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WHAT IS A POLICY TOOL? 


An Overview of the Tools Approach 
to Public Policy 


Michael Howlett 


Policy tools or instruments are techniques of governance that are used to give effect to stated 
policy objectives. The new design orientation within the policy sciences has placed renewed 
emphasis on policy tools and their role in achieving stated policy goals. This chapter provides an 
introductory overview of policy tools and traces the evolution of the tools approach in policy 
sciences. It begins with the distinction between first- and second-generation scholarship on 
policy tools and highlights their implications. This is followed by an overview of recent theoreti- 
cal and sector applications of the tools approach. The concluding discussion outlines avenues to 
further theorise the tools approach to public policy. 


Introduction: Policy Tools in Theory and Practice 


Policy instruments are techniques of governance which, one way or another, involve the utilisa- 
tion of state authority or its conscious limitation in order to achieve government aims. Hence, 
they properly fall within the domains of both public policy and political science but since they 
also often affect the behaviour of individuals in society as they go about their daily tasks, fall 
within the realms of other fields such psychology, gender studies, sociology and economics, 
among others. 

The policy alternatives which policymakers use are composed of different sets or combi- 
nations of the policy elements. As Linder and Peters noted, policy instruments are especially 
significant in policy designs and designing as they are the techniques or means through which 
states attempt to attain their goals. But tools are the subject of deliberation and activity at all 
stages of the policy process and affect both the agenda-setting and policy-formulation processes 
as well as being the subject of decision-making policy implementation and evaluation (Howlett 
2005; Howlett et al. 2009). 

Other terms have been developed in the field of policy studies to describe the same phenom- 
enon, such as ‘governing instruments’, ‘policy tools’ and the ‘tools of government’, and while 
these sometimes are used to refer to different mechanisms and calibrations of policy means, 
they are more often used synonymously. And it is also the case that most policy objectives can, 
in theory, be accomplished by a number of instruments; in other words, most instruments are 
to some degree ‘substitutable’. Thus, in theory, a government seeking to promote health care 
for the population could leave it entirely to the family to provide health services, with the 
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competence and availability of family members determining who gets how much and at what 
cost. Or the government might go to the other extreme and provide health services through 
its own administrative agency, paid for directly out of its general tax revenues, leaving no room 
for the market or other private organizations. In between the two extremes lie a range of other 
instruments, including exhorting the population to keep healthy, subsidizing those who are poor 
and regulating doctors and hospitals (Salamon 1989; Bemelmans-Videc et al. 1998). 

Policy tools have a special place in the consideration and study of policy design and policy 
implementation, however, because taken together, they comprise the contents of the toolbox 
from which governments must choose in building or creating public policies. Policy design 
elevates the analysis and practice of policy instrument choice — specifically, tools for policy 
implementation — to a central focus of study, making their understanding and analysis a key 
design concern (Salamon 1981; Linder and Peters 1990). Instrument choice from this perspec- 
tive is, in a sense, public policymaking, and understanding and analyzing potential instrument 
choices involved in implementation activity is policy design. 

Early students of policymaking had fairly flexible notions of the multiple means by which 
governments can affect, or give effect to, policy (Dahl and Lindblom 1953; Kirschen et al. 
1964; Edelman 1964; Lowi 1966). In his pathbreaking early works on public policymaking, for 
example, Harold Lasswell conceived the main instruments of politics as involving, among other 
things, the manipulation of symbols, signs and icons. Lasswell noted the extent to which govern- 
ments could affect each stage of the policy process through such manipulations and argued that a 
principal task of the policy sciences must be to understand the nuances of these actions and their 
effects (Lasswell 1954, 1971; Doern and Phidd 1988). 

By the early 1980s, at the urging of Lester Salamon and others, attention began to be focused 
on more precisely categorising policy instruments in order to better analyse the reasons for their 
use (Salamon 1981). Careful examination of instruments and instrument choices, it was argued, 
would not only lead to considerable insight into the factors driving the policy process and the 
characterisation of long-term patterns of public policymaking but would also allow practitioners 
to more readily draw lessons from the experiences of others with the use of particular techniques 
in specific circumstances (Woodside 1986). 

In most cases, policymakers use a mix of instruments in order to try to achieve their desired 
ends. This raises the questions of: (1) why specific mixes exist at present and (2) whether and to 
what extent mixes can be designed to be optimally effective, or, to put it another way, what are 
the constraints and impediments blocking optimal or efficient instrument use? 


Policy Tools Versus Implementation Tools 


It is important to note that policy instruments exist at all stages of the policy process, with 
specific tools such as stakeholder consultations and government reviews intricately linked to 
agenda-setting activities, ones like legislative rules and norms linked to decision-making behav- 
iour and outcomes, and others linked to policy evaluation, such as the use of ex-post, or after- 
the-fact, cost-benefit analyses. 

Policy tools are thus, in a sense, ‘multi-purpose’ since, for example, regulation can appear in 
the implementation activities of several governance modes while some tools, like impact assess- 
ments, can also appear within several stages of the cycle. However, a regulation appearing within 
the implementation phase of a network mode of governance which mandates information dis- 
closure, for example, serves a different purpose than a regulation found in a market mode which 
limits a firm to ownership of only a specific percentage of an industry. Similarly, consultations 
which take place in the agenda-setting stage of the policy process have a different purpose and 
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effect than those which take place after a decision has been made. While the general terminol- 
ogy may be similar, pains must be taken to distinguish these tools and activities in order to avoid 
confusion and errant efforts at instrument selection and policy design. 

As mentioned earlier, policy instruments appear in all stages of the policy process, and those 
affecting the agenda-setting, decision-making and evaluation stages of the policy process are very 
significant and important in public management (Wu et al. 2010). However, most policy designs 
deal with plans for implementation, and thus, most visible sets of policy instruments are those 
linked to policy implementation in the first instance and to policy formulation in the second. In 
the first category, we would find examples of many well-known governing tools such as public 
enterprises and regulatory agencies, which are expected to alter or affect the delivery of goods 
and services to the public and government (Salamon 2002), while in the second, we would find 
instruments such as regulatory impact or environmental impact appraisals, which are designed 
to alter and affect some aspect of the nature of policy deliberations and the consideration and 
assessment of alternatives (Turnpenny et al. 2009). 


Substantive Versus Procedural Policy Tools 


The systematic study of policy instruments in the 1970s and 1980s quickly generated a large 
academic literature and resulted in new policy initiatives in areas such as pollution prevention 
and professional regulation, which marshalled existing knowledge of tool use in other areas to 
these new purposes (Hippes 1988; Trebilcock 1983). Studies in Canada and elsewhere generated 
useful taxonomies of implementation tools in particular (Tupper and Doern 1981; Hood 1986; 
Vedung 1997; Howlett 1991) and shed light on significant subjects such as the reasons behind 
shifts in patterns of instrument choices associated with the waves of privatisation and deregula- 
tion which characterized the period (Howlett and Ramesh 1993). 

Much less attention was paid by analysts of this period to the systematic analysis of other 
kinds of tools, such as procedural ones, which featured at other stages of policymaking from 
agenda-setting to decision-making and evaluation (Howlett 2011). Although many early works 
defined ‘policy instruments’ broadly, so as to include a wide range of tools or techniques in 
these areas, by the 1980s, this larger set of tools had largely been forgotten or ignored in studies 
on the subject. In the 1990s, however, this neglect had been noted, and systematic treatments 
of procedural instruments began to emerge (Hood 1995, 1991; Dunleavy and Hood 1994; 
Riker 1986, 1983; Dunsire 1993a, 1993b, 1986) so that knowledge of both these types of 
instruments, their effects and the reasons behind their choices is now very much on par with 
that of implementation tools. 


Basic Taxonomies of Policy Instruments 


The key advantage which studies of policy instruments bring to policy studies is the limitation of 
policy outputs to only a few, manageable types, with variations in usage across different govern- 
ments and policy sectors illuminated and susceptible to careful analysis and generalisation. This 
not only sheds light on policy dynamics and the reasons why certain tools are utilised and not 
others, but also aids policy practice in allowing lessons to be drawn about ‘what works’ and why. 

Understanding implementation tools is thus key to policy design. One common type of 
implementation instrument alters the actual substance of the kinds of day-to-day production, 
distribution and consumption activity carried out in society, while the other focuses on alter- 
ing political or policy behaviour in the process of the articulation of implementation goals and 
means. Substantive implementation instruments are those used to directly affect the production, 
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distribution and consumption of goods and services in society while procedural implementation 
instruments accomplish the second purpose. 

This distinction is apparent in common definitions of governing instruments, although its 
significance is sometimes overlooked. Vedung, for example, has usefully defined policy instru- 
ments used in implementation activities as ‘the set of techniques by which governmental author- 
ities wield their power in attempting to ensure support and effect social change’ (Vedung 1997). 
This definition can be seen to include both ‘substantive’ tools, those Hood (1986) defined as 
attempting to ‘effect or detect’ change in the socio-economic system, and those ‘procedural’ 
tools designed to “ensure support’ for government actions. 

Substantive instruments are expected to alter some aspect of the production, distribution and 
delivery of goods and services in society. This is a large field of action since it extends not only 
to goods and services provided or affected by markets but also well beyond to state or public 
provision and regulation, as well as to those goods and services typically provided by the family, 
community, non-profit and voluntary means, often with neither a firm market nor a state basis 
(Salamon 1989, 2002), broadly conceived to include both mundane goods and services like 
school lunches to crude vices such as gambling or illicit drug use to more common individual 
virtues such as charitable giving or volunteer work with the physically challenged, and include 
the attainment of sublime collective goals like peace and security, sustainability, happiness and 
well-being. 

Substantive implementation instruments can affect many aspects of production, distribution 
and consumption of goods and services regardless of their institutional basis. Production effects, 
for example, include determining or influencing: 


1. Who produces it — for example, via licencing, bureaucracy/procurement or subsidies for 
new start-ups. 

2. The types of goods and services produced — for example, through bans or limits or 
encouragement. 

3. The quantity of goods or services provided — for example, via subsidies or quotas. 

4. The quality of goods or services produced — for example, via product standards, warranties. 

5. Methods of production — for example, via environmental standards or subsidies for 
modernization. 

6. Conditions of production — for example, via health and safety standards, employment stan- 
dards act, minimum wage laws, inspections. 

7. The organization of production — for example, via unionization rules, antitrust or anti- 
combines legislation, securities legislation or tax laws. 


Consumption and distribution effects are also manifold. Some examples of these are: 


1. Prices of goods and services — such as regulated taxi fares or wartime rationing. 

Actual distribution of produced goods and services — affecting the location and types of 
schools or hospitals, forest tenures or leases. 

3. Level of consumer demand for specific goods — for example, through information release, 
nutritional and dangerous goods labelling (cigarettes), export and import taxes and bans and 
similar activities. 

4. Level of consumer demand in general — via interest rate, monetary and fiscal policy. 


In the case of substantive policy instruments, or those instruments intended to directly affect 
the nature, types, quantities and distribution of the goods and services provided in society, a 
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great deal of conceptual progress has occurred over the past several decades (Salamon 2002). 
Taxonomies, for example, have been provided by many authors, one of the most well known 
developed by Christopher Hood (1986; see also Anderson 1977). In this scheme, instruments 
are grouped together according to (1) whether they rely upon the use of ‘nodality’ (or informa- 
tion), authority, treasure or the organizational resources of government for their effectiveness 
and (2) whether the instrument is designed to effect a change in a policy environment or detect 
changes in it. A typical taxonomy of substantive policy instruments based on Hood’s schema is 
presented in Table 1.1. 

Procedural policy instruments, as noted here, have been studied less systematically, although 
many studies of individual tool use exist (Landry et al. 1998). The works of Bressers and Klok 
(1988), Schneider and Ingram (1990a, 1990b), and others (McDonnell and Elmore 1987; 
Elmore 1978, 1987) in the 1980s and 1990s, for example, identified a large number of typi- 
cal procedural policy instruments. These include education, training, institution creation, the 
selective provision of information, formal evaluations, hearings and institutional reform (Weiss 
and Tschirhart 1994; Bellehumeur 1997; Chapman 1973; Wraith and Lamb 1971; Peters 1992; 
Kernaghan 1985). Research into the tools and mechanisms used in intergovernmental regula- 
tory design identified several other such instruments, including ‘treaties’ and a variety of ‘politi- 
cal agreements’, which can affect target group recognition of government intentions and vice 
versa (Bulmer 1993; Harrison 1999; Doern and Wilks 1998). Other research into interest group 
behaviour and activities highlighted the existence of tools related to group creation and manipu- 
lation, including the role played by private or public sector patrons in aiding the formation and 
activities of such groups (Pal 1993; Burt 1990; Finkle et al. 1994; Phillips 1991; Nownes and 
Neeley 1996; Lowry 1999). Still other specialised research into aspects of contemporary poli- 
cymaking has highlighted the use of techniques such as provision of research funding for, and 
access to, investigative hearings and tribunals (Gormley 1989; Jenson 1994; Cairns 1990; Salter 
and Slaco 1981). 

Procedurally oriented implementation tools affect production, consumption and distribution 
processes only indirectly, if at all. Rather, they instead affect the behaviour of actors involved in 
policy implementation. Policy actors are arrayed in various kinds of policy communities, and 
just as they can alter or affect the actions of citizens in the productive realm, so, too, can they 
affect and alter aspects of policy-making behaviour. Procedural implementation tools are an 


Table 1.1 A Taxonomy of Substantive Policy Instruments 


Principal Governing Resource Used 


Nodality Authority Treasure Organization 
Detector 
General Advice Regulation Grants Bureaucratic 
Purpose of Training Licences User Charges Administration 
Instrument Taxes Public Enterprises 
Loans 
Effector 
Reporting Census Taking Polling Record Keeping 
Surveys Registration Consultants Police Reporting 


(Cells provide examples of instruments in each category.) 


Source: Adapted from Hood (1986) 
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important part of government activities aimed at altering policy interaction within policy sub- 
systems, but, as Klijn et al. (1995) put it, they typically ‘structure . . . the game without deter- 
mining its outcome’ (441). That is, these behavioural modifications affect the manner in which 
implementation unfolds but without predetermining the results of substantive implementation 
activities. 

Procedural implementation tools and their effects are not as well studied or understood as are 
substantive instruments, although several procedural techniques, such as the use of specialised 
investigatory commissions and government reorganizations, are quite old and well used and have 
been the objects of study in fields such as public administration, public management and orga- 
nizational behaviour. Nevertheless, just like their substantive counterparts, they are a key part of 
policy designs and policy design activity. 

Some of the kinds of implementation-related activities that can be affected by the use of 
procedural tools (Klijn et al. 1995; Goldsmith and Eggers 2004; Klijn and Koppenjan 2006) 
include: 


Changing actor policy positions. 

Setting down, defining or refining actor positions. 

Adding actors to policy networks. 

Changing access rules for actors to governments and networks. 
Influencing network formation. 

Promoting network self-regulation. 

Modifying system-level policy parameters (e.g., levels of market reliance). 
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Changing evaluative criteria for assessing policy outcomes, success and failure. 
9. Influencing the payoff structure for policy actors. 
10. Influencing professional and other codes of conduct affecting policy actor behaviour. 
11. Regulating inter-actor policy conflict. 
12. Changing policy actors’ interaction procedures. 
13. Certifying or sanctioning certain types of policy-relevant behaviour. 
14. Changing supervisory relations between actors. 


Hood’s taxonomy of substantive instruments can be modified to help make sense of this dis- 
parate (and partial) inventory of procedural tools. That is, classifying procedural instruments in 
accordance with the type of ‘governing resource’ they rely on generates a useful preliminary 
taxonomy (see Table 1.2) (Saward 1992). While most researchers have focused on the manner 
in which these instruments have been used to enhance participation and policy-relevant knowl- 
edge, it should also be emphasised that procedural tools can also be used to negatively affect 
interest groups and other actor behaviour. That is, for example, information-based procedural 
instruments include both the provision of information and its suppression, and the release of 
misleading as well as accurate information. Deception, obfuscation and other forms of admin- 
istrative delay, similarly, are all forms of authority-based procedural instruments (Mueller 1973). 
Hence, drawing a distinction between ‘positive’ and ‘negative’ uses of governing resources in 
terms of whether they encourage of discourage actor participation in policy processes is a useful 
aspect of the preliminary classification of such instruments. 

As was the case with substantive instruments, this taxonomy is useful insofar as it highlights 
the different basic resources used by different types of instruments and therefore allows a virtu- 
ally unlimited number of instruments to be placed in a limited number of general categories, 
facilitating analysis and helping improve policy practice. 
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Table 1.2 A Resource-Based Taxonomy of Procedural Policy Instruments 


Principal Governing Resource Used 


Nodality Authority Treasure Organization 
Positive Education Labelling Interest Group Institutional 
Information Treaties and Political Creation Reform 
Provision Agreements Intervenor Judicial 
General Focus Groups Advisory Group and Research Review 
Purpose of Creation Funding Conferences 
Instrument 
Negative Propaganda Banning Groups and Eliminating Administrative 
Information Associations Funding Delay and 
Suppression Obfuscation 


Denial of Access 


(Cells provide examples of instruments in each category.) 


The Rationale for Instrument Use: Market and Subsystem Manipulation 


Understanding these basic types of policy instruments and their permutations accomplished one 
of the main goals of students of policy instruments in the post-Salamon era. However, simply 
describing the nature of the instruments available to policymakers was only the first step towards 
providing better advice to those same policymakers about which instruments to choose in which 
circumstance. That is, the aim is not only better description but also better prescription, and in 
order to accomplish this, it is necessary to elevate the discussion from taxonomies to the analysis 
of the reasons for tool adoption, or what is usually referred to in the literature as ‘the rationale 
for instrument choice’. 

Towards this end, many analysts at various points in time proposed various schemes which 
purported to establish the relationship existing between different instruments or categories 
of instruments and the successful attainment of government objectives (Doern and Aucoin 
1971; Doern and Wilson 1974; Tupper and Doern 1981). For example, developed a spectrum 
of substantive instruments based on Hood’s taxonomy. They focused on the level of direct 
state involvement in the provision of goods and services as the chief criterion for distinguish- 
ing between categories of ‘effector’ instruments. This placed ‘voluntary’ instruments requiring 
minimal state involvement at one end of a continuum with state-based instruments such as 
public enterprises placed at the opposite end. Between the two poles lie a wide range of ‘mixed’ 
instruments involving varying levels of state and private provision of goods and services (see 
Figure 1.1). 

Much the same can be done for procedural instruments. That is, as Dutch scholars such as 
Klijn, Kickert, Koppenjan and especially de Bruijn and ten Heuvelhof have argued, procedural 
policy instruments can be thought of as involving the manipulation not of economic exchange 
relationships, as is the case with substantive tools, but rather of the links and nodes of the net- 
work relationships existing among actors involved in policymaking Klijn 1996; de Bruijn and 
ten Heuvelhof 1995, 1997; Leik 1992). Construed in this way, procedural instruments can be 
seen to be used to manipulate the number or nature of actors arrayed in the policy subsystems 
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Level of State Provision of Goods and Services 


Low  <——— = _ High 
Voluntary Mixed Compulsory 


Figure 1.1 A Spectrum of Substantive Policy Instruments 


Source: Howlett and Ramesh (1995) 


Information Funding Recognition Institutional 


Level of State Manipulation of Subsystem 
Low = High 
(Management) (Restructuring) 
Voluntary Mixed Compulsory 


Figure 1.2 A Spectrum of Procedural Policy Instruments 


Source: Howlett (2000) 


which policymakers face, each category of instrument using a specific resource in order to 
manipulate an aspect of a policy subsystem or network. 

As de Bruijn and ten Heuvelhof (1995, 1991; see also Klijn and Teisman 1991; Peters 1998) 
have pointed out, a wide range of activities are possible in network manipulation, ranging from 
limited ‘network management’ to more fundamental ‘subsystem restructuring’. Incorporating 
this distinction allows the procedural policy instruments found in Table 1.2 to be arrayed in 
a single spectrum according to the level of state manipulation of subsystem membership and 
activities (see Figure 1.2). 

In this spectrum, procedural policy instruments can be seen to range from limited informa- 
tion suppression or release designed to mildly affect subsystem behaviour through ‘voluntaristic’ 
responses from targeted actors to group or institutional reforms designed to completely restruc- 
ture existing subsystems by compulsory means (Smith et al. 1993; Savoie 1999). 


The Variables Affecting Instrument Choice: State Capacity 
and Target Complexity 


It is only a start, however, to say that a variety of instrument choices exists which can alter pat- 
terns of goods and service delivery or policy interactions and that these choices differ in terms 
of the extent of state involvement in them. Rather, what is needed is the identification of a 
limited set of factors or variables which can be said to influence instrument choices in specific 
directions. 

In a perfect world, there would be little trouble choosing the appropriate tool for the 
governmental task at hand. That is, if all the costs and benefits of a tool were context-free 
and known, and the goals of a policy clear and unambiguous, then a decision on which 
instrument to use in a given circumstance would be a simple maximising one, and mistakes 
would not be made. However, in real-world situations, as information difficulties arise in 
determining instrument effects and as the clarity and precision of goals diminish, it becomes 
more and more likely that policy means and ends will be mismatched and policy failures 
occur. 

Moreover, it has become more and more apparent to many observers that the kind of preci- 
sion required for such maximising instrument choices will never be achieved, not just because 
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of poorly defined, ambiguous decision-making circumstances and information asymmetries but, 
more fundamentally, because the utility of the instruments themselves, and hence the calculation 
of their attractiveness, is heavily context dependent. That is, although instruments may be, in 
some technical or theoretical sense, ‘substitutable’, in the sense that any one could achieve any 
end — albeit at differing levels of cost — in practice, they differ in a number of ways which makes 
the choice of instrument a complex matter. 

Salamon and Lund, for example, suggest that different instruments involve varying degrees 
of effectiveness, efficiency, equity, legitimacy and partisan support, which affect their appropri- 
ateness for a particular situation (Salamon and Lund 1989). Thus, some instruments are more 
effective in carrying out a policy in some contexts than others. Efficiency, for example, in terms 
of low levels of financial and personnel costs, may be an important consideration in climates 
of budgetary restraint but a less significant aspect in free-spending times. Legitimacy is another 
critical aspect of instrument use which varies with context (Beetham 1991; Suchman 1995). 
The ability of an instrument to attract the support of the population in general and, particularly, 
of those directly involved in policymaking in the issue area or subsystem involved must also be 
taken into account. A relatively heavy-handed approach to regulation of the financial dealings 
of industry, for example, may be anathema in normal times in many sectors, but in the wake 
of bank failures or scandals, such as the Enron affair in the US or Parmalat in Europe, may find 
sudden popularity among both the public and policy elites. Abstract notions of efficiency and 
effectiveness may also find themselves less important criteria in some contexts, like wartime, 
when the use of government departments or public enterprises, simply because they remain 
under direct government control (Borins 1982) Vining and Botterell 1983) or because admin- 
istrators may be more familiar with their use and risks, are preferred tool choices (May 1993; 
Hawkins and Thomas 1989). Moreover, cultural norms and institutional or political arrange- 
ments may accord greater legitimacy to some instruments than others. Thus, it is possible that in 
liberal democracies, citizens and policymakers desiring high levels of individual autonomy and 
responsibility may prefer instruments that are less coercive rather than other equally or perhaps 
more effective or efficient alternatives which do not promote these values. Such societies can be 
expected, for example, to prefer voluntary and mixed instruments to compulsory instruments 
on philosophical or ideological grounds (Doern and Wilson 1974; Doern 1974; Howlett 1991). 
Moreover, instruments have varying distributional effects, and so policymakers in such societies 
may need to select instruments that are, or at least appear to be, equitable. Tax incentives, for 
example, are inherently inequitable because they offer no benefit to those (the poor) without 
taxable income. Their use, therefore, will vary to the extent that (1) societies are bifurcated along 
socio-economic or class lines, and (2) individuals are aware of their advantageous and pernicious 
consequences. 

In addition to these ‘external’ contexts, there are also ‘internal’ constraints on instrument 
choices that must be considered. That is, while instrument choice is clearly not a simple techni- 
cal exercise and must take into account aspects of the social, political and economic contexts of 
instrument selection, it is also the case that the internal configuration of instrument mixes alters 
the calculus of instrument choice in significant ways. That is, some instruments may work well 
with others — as is the case with ‘self-regulation’ set within a regulatory compliance framework 
(Gibson 1999; Grabosky 1995; Trebilcock et al. 1979; Tuohy and Wolfson 1978) — while other 
combinations — such as, notably, independently developed subsidies and regulation (de Moor 
1997; Myers and Kent 2001) — may not. 

Both these ‘internal’ and ‘external’ contexts of instrument behaviour and selection must be 
taken into account in efforts to theorise optimality in the design of policy mixes (Minogue 
2002). Unfortunately, however, this is often not the case, and instrument choices are often 
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viewed through an ideological or conceptual lens which reduces choices to a ‘one size fits all’ 
motif or, more commonly, to a struggle between ‘good and evil’ in which an existing range of 
instrument uses is condemned and the merits of some alternative single instrument trumpeted 
as the embodiment of all that is good in the world. The unfortunate consequence of such 
approaches, if adopted, is usually to wield that instrument — be it state-driven public enterprises 
in the case of socialist and developing countries in the first two-thirds of the twentieth century 
or the virtues of privatisation, deregulation and markets in the last third — less like the scalpel 
of a careful surgeon working on the body politic and more like the butcher’s cleaver, with little 
respect for the tissue of the patient falling under the knife. 

Howlett and Ramesh, again, provide an example of how these two variables and their 
expected relationship to each other can be used to generate a simple model containing a set of 
hypotheses regarding substantive instrument choices (see Table 1.3). 

In this model, for example, it is argued that subsidy or market instruments should only be 
used, or can only be used effectively, when a high level of state capacity and a complex policy 
subsystem exists — as is the case, for example, with most competitive economic situations faced 
by modern states. If a state faces a complex network or subsystem but has only limited capac- 
ity, on the other hand, it is expected that it will tend to utilise regulatory or information-based 
instruments. Direct provision and public enterprises would be expected to be used only when 
a state has high capacity but faces a relatively simple social or policy environment characterised 
by few actors and a small number of significant interorganizational relationships. Finally, when 
state capacity is low and the policy environment not very complex, reliance on voluntary 
instruments can be effective, as was the case historically in many areas of social and health pol- 
icy (Vogel 1996; Eisner 1994; Tupper 1979; Laux and Molot 1988; Hall and Banting 2000). 

These kinds of models do not delve into the detail of fine gradations of instrument use 
within each general category or the specific contexts of individual decisions which can result 
in errors being made in instrument choices. However, they suggest that although substantive 
instrument choices are complex, general patterns of such choices can nevertheless be discerned 
and explained. The development of instrument taxonomies and spectra and the formulation of 
bivariate conceptual models helped identify these patterns and the limited number of variables 
responsible for them. 

This much is well known about instruments. However, in practice, as was pointed out earlier, 
most governments do not use single instruments to address problems but usually adopt a variety 
of tools to accomplish their ends. While these mixes can become quite complex — either by 
design or by accident of history — it is worth noting that although seemingly faced with a wide 


Table 1.3 A Model of Policy Instrument Choice 


Level of Policy Subsystem Complexity 


Level of State Capacity High Low 

High Market or Subsidy Direct Provision 
Instruments Instruments 

Low Regulatory or Voluntary, Community or Family-Based 
Information Instruments 
Instruments 


(Cells indicate likely instrument choice.) 


Source: Howlett and Ramesh (1995) 
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choice of possible instruments in creating their strategies, governments often repeatedly choose 
from a much more limited set of options. That is, there is a distinct tendency for governments 
to develop an ‘implementation style’ in various sectors and to stick with that style for quite some 
time (Kagan and Axelrad 1997; Kagan 1997). 

An emphasis on long-standing patterns of instrument choices is not to say, of course, that 
choices are inevitable or immutable or that substantial shifts in implementation styles do not 
occur. These can happen as the nature of the constraints governments face changes or if the 
governments decide to broaden or narrow their focus on specific policy targets. This is a central 
theme of this chapter, of course — embodied in the title. Assessing how likely it is for existing 
implementation styles to change, therefore, is an important question. 

Certainly, shifts in fundamental implementation styles have occurred in many governments 
over the past century due to the influences of activities like colonisation and decolonisation, 
war, and other events which have wrought changes to the organizational capacities of states 
and their societies. Even in governments less affected by such dramatic events, such as those in 
North America, implementation styles have moved from, for example, a preference for directed 
subsidisation in the nineteenth century and then to the regulatory corporatism associated with 
the progressive movement in the 1920s and 1930s (Raadschelders 1998, 2000; Eisner 1994; 
Lowi 1966). 

It is also certainly the case in Europe, North America and elsewhere that contemporary 
governance takes place within a very different context from that of past decades. Government 
capacity in terms of human and organisational resources remains high by historical standards, 
but the autonomy or ability of governments to independently affect change has been eroded 
by such factors as the growth of powerful international actors and systems of exchange (Cerny 
1996). Moreover, at the domestic level, modern societies have developed increasingly complex 
networks of interorganisational actors whose coordination and management are increasingly 
problematic (Lehmbruch 1991; Mayntz 1993). 


Conclusion: The Need for Nuance and Precision in 
Policy Tool Analysis and Practice 


The study of policy instruments over the past 50 years has generated many insights into instru- 
ment use: insights which have helped academics better understand policy processes and have 
helped practitioners in Canada and elsewhere design better policies (Gibson 1999; Weimer 
1992). However, in the process of developing the taxonomies and models of instrument choice, 
many investigators have focused almost exclusively on the specific set of instruments which 
governments use to alter the distribution of goods and services in society. In focusing so intently 
on ‘substantive’ policy instruments, sight has been lost of the need, identified by early students 
of public policy, to take both the substance and process of policymaking into account when 
conducting instrument analyses. 

This has become a major problem in attempting to find solutions, methods and tools to deal 
with contemporary policy issues (Kooiman 1993; Peters and Pierre 1998; Klijn and Koppenjan 
2000; Walters et al. 2000). At the present time, however, the basic contours of both procedural 
and substantive policy instruments are well known, as are their basic rationales for use and 
the factors which have led to their selection. Although understanding the use of ‘bundles’ or 
‘portfolios’ of instruments rather than single tools is crucial in designing effective governance 
strategies (Doremus 2003; Sterner 2002), the concept of an ‘implementation style’ is a useful 
one in beginning to assess this question. All these issues, and more, are addressed in the chapters 
contained in this handbook. 
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THE HISTORY OF THE TOOLS 
APPROACH IN THE POLICY 
SCIENCES 


Key Definitions and Problematics 


Michael Howlett 


Understanding the range of possibilities present in any design situation is a key factor for both 
policy advisors and decision-makers and requires an understanding of what kinds of instrument 
options exist, which subset of those is generally considered feasible or possible in a given con- 
text, and which among that smaller subset of all possible tools is the most appropriate to use at 
a given time. In the effort to help deal with these questions, students of policy formation and 
policy instruments in a variety of academic disciplines have, over the years, developed several 
models or conceptual schemes which help explain how policy instruments differ and how the 
different ones can be used in specific circumstances to achieve specific kinds of ends. This study 
of the “tools of government” has a long history and a rich tradition of research in the policy 
sciences, much of it borrowed from public administration, law and regulatory studies, public 
management, and many sectoral fields, from telecommunications and environmental regulation 
to social policy and education studies. The main findings and currents of thinking in this area 
are set out in this chapter. 


Introduction 


A significant part of policy-making activity involves matching policy goals with the ideas for- 
mulators hold about feasible and desirable mixes of policy means. Understanding the range of 
possibilities present in any policy-making situation is a key factor for both policy advisors and 
decision-makers and requires an understanding of what kinds of instrument options exist, which 
subset of those is generally considered feasible or possible in a given context, and which among 
that smaller subset of tools is the most appropriate to use at a given time. 

In the effort to help deal with these questions, students of policy formation and policy instru- 
ments in a variety of academic disciplines have, over the years, developed several models or con- 
ceptual schemes which help explain how policy instruments differ and how the different ones 
can be used in specific circumstances to achieve specific kinds of ends. This study of the “tools of 
government” has a long history and a rich tradition of research in the policy sciences, much of it 
borrowed from public administration, law and regulatory studies, public management, and many 
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sectoral fields, from telecommunications and environmental regulation to social policy and edu- 
cation studies. The main findings and currents of thinking in this area are set out in this chapter. 


The Origins of Policy Instrument Study as Field of Academic Inquiry 


Fields interested in studying public policy, such as political science and political sociology, have 
traditionally been concerned with studying policy “inputs” or the dynamics of public policy 
formation. For example, in political science, a key focus has been on the role played by public 
opinion, political party activities, elections, and similar phenomena in affecting policy-making 
processes and defining policy content while, in the case of political sociology, a key focus has 
been on understanding the roles played by social structure in defining actor “interests” and posi- 
tions in policy-making processes (Mayntz 1983). 

Studies in these disciplines have revealed a great deal about policy-formation processes but 
tended to neglect the implementation component of policymaking. Studies in other fields, such 
as public administration and management and organization studies, on the other hand, have 
traditionally focused their efforts on the study of the inner workings of government, especially 
the study of behavioural and management issues involved in such tasks as financial administra- 
tion and budgeting, ministerial responsibility and accountability, the operation of the merit 
principle, and human resources/personnel administration. While these studies often purposely 
avoided considering the more political aspects of policy processes, they provided a great deal of 
information on implementation issues, which has helped fill in the blanks left in earlier studies. 

In his pathbreaking early works on public policymaking, Harold Lasswell drew on both these 
literatures, not only to define public policy, clarify important aspects of policymaking such as 
the number and type of stages involved in policy deliberations, and emphasize the importance 
of context to its workings (Torgerson 1985, 1990) but also to think about the main instruments 
of policymaking. 

Lasswell (1954) noted the extent to which governments could affect policymaking through 
manipulations involving, among other things, “symbols, signs and icons”, and argued that a 
principal task of the policy sciences must be to understand the nuances of these actions and their 
effects (Lasswell 1954, 1971). Like others of Lasswell’s insights, this orientation was retained by 
many later students of policymaking who developed very flexible notions of the multiple means 
by which governments could affect, or give effect to, policy. 

In these early works, “policy instruments” were defined very broadly so as to include a wide 
range of tools or techniques of governance used at different stages of the policy process. How- 
ever, in the 1970s, as the effort to improve policymaking through improved policy designs took 
shape, work turned to focus on the evaluation of the impact on policy outcomes of specific 
kinds of implementation related tools, primarily economic ones like subsidies and taxes (Mayntz 
1983; Woodside 1986; Sterner 2003) but later extended to many more types and variations. 

Many authors and scholars, following this lead, argued for a fundamental recasting of policy 
studies along the lines of implementation research. Bardach (1980) and Salamon (1981), for 
example, both argued in the early 1980s that policy studies had “gone wrong” right at the start 
by defining policy in terms of “issues”, “areas” or “fields”, rather than in terms of “instruments”. 
As Salamon put it: 


The major shortcoming of current implementation research is that it focuses on the 
wrong unit of analysis, and the most important theoretical breakthrough would be 
to identify a more fruitful unit on which to focus analysis and research. In particular, 
rather than focusing on individual programs, as is now done, or even collections of 
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programs grouped according to major “purpose,” as is frequently proposed, the sug- 
gestion here is that we should concentrate instead on the generic tools of government 
action, on the “techniques” of social intervention. 

(1981: 256) 


Following these kinds of injunctions, other scholars began to investigate the links between 
implementation failures and policy success in more detail and turned their gaze directly on the 
subject of how implementation alternatives were crafted and formulated (Mayntz 1979; Goggin 
et al. 1990; O’ Toole 2000). Studies in economics and law which focused on the “ex-post” evalu- 
ation of the impact of policy outputs (Bobrow 1977; Stokey and Zeckhauser 1978), for example, 
began the more systematic appraisal of implementation alternatives. Many lessons about policy 
instruments and policy design were also drawn from legal studies, for example, which revealed a 
great deal about how tools such as laws, regulations, and other mechanisms involved in the deliv- 
ery of various kinds of goods and services operate and upon procedural aspects of formulation 
and implementation activities such as the passage of legislation and forms of administrative rule- 
making, while organization, management, and administrative studies provided insights into the 
links between administrative systems and governance modes, among others (Peters and Pierre 
1998; Pierre and Peters 2005). Ultimately, insights gleaned from a wide body of interdisciplinary 
literature concerning policy inputs and governmental processes were combined in the 1980s and 
1990s in the explicit study of policy instruments and their role in policy design. 

Studies from the early 1980s focused on the need to more precisely categorize types of policy 
instruments in order to better analyze the reasons for their use (Salamon 1981; Tupper and 
Doern 1981; Trebilcock and Hartle 1982; Bressers and Honigh 1986; Bressers and Klok 1988). 
Careful examination and systematic classification of implementation instruments and instrument 
choices, it was argued, would not only lead to insights into the factors driving the policy process 
and the characterization of long-term patterns of public policymaking, as Lasswell had hoped, 
but would also allow practitioners to more readily draw lessons from the experiences of others 
with the use of particular techniques in specific circumstances and hence improve policy designs 
and outcomes (Mayntz 1983; Linder and Peters 1984; Woodside 1986). 

During this period, studies in Europe and North America shed a great deal of light on 
the construction and establishment of regulatory and other political and administrative agen- 
cies and enterprises, traditional financial inducements, and the “command-and-control” mea- 
sures adopted by administrative agencies (Tupper and Doern 1981; Hood 1986; Howlett 1991; 
Vedung 1998; Landry et al. 1998). And this new emphasis on the systematic study of policy 
instruments quickly generated a sizable academic literature and resulted in immediate applica- 
tion in the design of many new policy initiatives in emerging areas such as pollution prevention 
and professional regulation (Trebilcock 1983; Hippes 1988). Significant subjects such as the 
reasons behind shifts in patterns of instrument choices associated with the waves of privatization 
and deregulation which characterized the period also received attention (Howlett and Ramesh 
1993). 

Most of these studies, however, focused exclusively on what were referred to in previous 
chapters as “substantive instruments”: that is, those which directly affect the production and 
delivery of goods and services in society. These early studies failed to adequately address proce- 
dural tools and consequently, until around the year 2000, developed only a partial description of 
policy tools and an understanding of how instrument choices related to policy design. 

Nevertheless, by the late 1980s the field of instrument studies had advanced enough that 
Salamon (1989) could argue that the “tools approach” had become a major approach to policy 
studies in its own right, bringing a unique perspective to the policy sciences with its focus on 
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policy outputs. Salamon argued that this perspective had revealed that not only did, as traditional 
studies had maintained, “politics determine policy”, but also the reverse (Landry et al. 1998). 
That is, via the feedback mechanism in the policy cycle (Pierson 1992, 1993), tool choices led to 
the establishment of a “political economy” of a policy regime: a tool choice such as, for example, 
a decision to use tax incentives to accomplish some end, created a constituency for continuation 
of that incentive (and sometimes one opposed to it), affecting future policy deliberations and 
decisions including those related to instrument choices (Linder and Peters 1984; Bobrow and 
Dryzek 1987; Dryzek and Ripley 1988). 

At this point, Salomon framed two important research questions to be addressed in future 
analyses of the tools of government action: “What consequences does the choice of tool of gov- 
ernment action have for the effectiveness and operation of a government program?” and “What 
factors influence the choice of program tools?” (265). These questions were taken up by the 
“tools approach” and the policy design literature in the 1990s. 


The Development of Models in the Study of Policy Tools 


Assessing and answering Salomon’s questions required scholars interested in policy design to 
engage in a lengthy process of social scientific analysis and model building related to the study 
of implementation tools. These efforts expanded the number of preliminary questions which 
needed to be answered before Salomon’s queries could be addressed (Salamon 1981; Timmer- 
mans et al. 1998; Hood 2007) to include: 


What potential tools does any government have? 
How can these be classified? 

How have these been chosen in the past? 

Is there a pattern for this use? 

If so, how can we explain this (or these) pattern(s)? 


OS OY Be Oh OE 


Can we improve on past patterns of use? 


The Construction of Empirical Inventories 


The first step in the systematic study of policy instruments, as in any other similar endeavour in 
the social sciences, is the establishment of an inventory of the dependent variable. 

While there were many scholars who had looked at specific tools in the past (such as Cush- 
man’s 1941 study of regulatory agencies, which was often cited by early students of the field), the 
first efforts to systematically define the range of possible instruments which could be used in a 
policy design originated in the post-World War II planning exercises undertaken by the United 
Nations and the Organization for Economic Cooperation and Development (OECD) in Europe. 

Key figures in this research included Nobel Prize-winning development economists such as 
E. S. (Etienne) Kirschen and Jan Tinbergen, who published groundbreaking studies including 
Economic Policy in Our Times (1964), dealing with the instruments for economic policy they had 
viewed in operation in the process of post-war European reconstruction. One of the first inven- 
tories of instruments was Kirschen et al’s (1964) identification of well over 40 different types 
of implementation instruments then prevalent in European economic policy-making activities, 
ranging from public enterprises to various forms of government procurement and tax incentive 
and subsidy schemes. 

Such studies were followed by many others examining the instruments prevalent in other 
areas, such as banking and foreign policy (Hermann 1982), adding to the list tools such as 
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interest rate determination and other monetary and fiscal tools. These were pathbreaking studies 
which, although they did not make any distinctions between general implementation prefer- 
ences, policy mechanisms, and calibrations, and very often confused implementation tools and 
instruments used at other stages of the policy process, laid the groundwork for such future 
refinements by providing the raw data required for later classification efforts. 


The Development of Taxonomies 


Once a fairly exhaustive inventory has been created, the next major step in theory construction 
is to move towards taxonomy: that is, examining the list of the phenomena under consideration 
and attempting to classify or categorize the subject matter into a smaller number of mutually 
exclusive categories, which together retain the exhaustive character of the original lists. Many 
such schemes were developed in the policy instruments literature of the 1960s to 1980s. 

Kirschen and his fellow authors (1964), for example, utilized a resource-based taxonomy of 
governing instruments to group instruments into five general “families” according to the “gov- 
erning resource” they used: public finance, money and credit, exchange rates, direct control, 
and changes in the institutional framework (16—17). However, this scheme was very sectorally 
specific and focused on the specific problem of achieving economic development goals. More 
generic schemes were developed, such as that put forward by Theodore Lowi (1966, 1972), 
which heavily influenced later thinking on the subject. 

Lowi developed the insight first put forward by students of public administration in the USA 
like Cushman (see Figure 3.2) that governments had only a small number of alternative choices 
in any given regulatory situation, depending on the amount of coercion they wished to employ 
in that situation — in Cushman’s case, choosing either to regulate or not and, if so, to regulate 
either by the use of public enterprises or regulatory commissions. This analysis, among other 
things, introduced the idea that instrument choices were multi-level and nested, an insight 
which would be further developed in the years to come. 

In his own work, however, Lowi argued that a four-cell matrix based on the specificity of 
the target of coercion and the likelihood of its actual application would suffice to distinguish 
all the major types of government implementation activity. The original three policy types he 
developed included the weakly sanctioned and individually targeted “distributive” policies, the 
individually targeted and strongly sanctioned “regulatory” policy, and the strongly sanctioned 
and generally targeted “redistributive” policy. To these three, Lowi later added the weakly sanc- 
tioned and generally targeted category of “constituent” policy (see Table 4.1). 

This was a significant advance, since it attempted to reduce the complexity of instrument 
choice to a single two-dimensional framework. However, Lowi’s categories of tools — distributive, 
redistributive, constituent, and regulatory — did not fit well with existing tool inventories and 
hence were difficult to operationalize and test (Roberts and Dean 1994). As a result, many other 
classification schemes emerged in the literature in the mid-to-late 1980s. 

Many of these efforts, but not all, followed Lowi’s and Cushman’s lead in focusing on some 
aspect of coercion as the key element differentiating policy instrument types. In a key develop- 
ment, however, some also introduced a greater number of differentiating criteria. Balch (1980), 
for example, talked about both “carrots” (inducements or incentives) and “sticks” (coercion or 
disincentives), while Bardach (1980) argued that government had three “technologies” which they 
could utilize in any given choice situation — enforcement, inducement, and benefaction — and that 
these strategies required different combinations of four critical governmental resources: money, 
political support, administrative competency, and creative leadership. Elmore (1987) identified 
four major classes of instruments: mandates, inducements, capacity building, and system changing. 
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Table 2.1 Hood’s 1986 Taxonomy of Substantive Policy Instruments 


Governing Resource 


Nodality Authority Tieasure Organization 
Principle Detectors Surveys Licencing Policing Record keeping 
Use Effectors Public information campaign Regulation Subsidies | Government agencies 


Source: Adapted from Christopher Hood, The Tools of Government, Chatham: Chatham House Publishers, 1986 


Further studies moving in the design direction refined this idea of only a limited number 
of “governing resources” lying behind each tool. Christopher Hood (1983, 1986) generated a 
major work on the subject in 1986 which was heavily influenced by detailed studies of the Brit- 
ish and German policy implementation processes undertaken previously by Dunsire (1978) and 
Mayntz (1975). It involved the elaboration of a fourfold resource-based categorization scheme 
for policy instruments which served as an admirable synthesis of the other, earlier, models. 

Hood argued that governments have essentially four resources at their disposal, which they 
can use to either effect changes in their environment or detect them: nodality, meaning the 
resource that existed simply by nature of the fact that governments existed at the “centre” of 
social and political networks but which can also be thought of as “information” or “knowl- 
edge”; authority; treasure; and organization (or “NATO” in Hood’s terminology). In Hood’s 
scheme, implementation instruments are grouped together according to which of the NATO 
resources they most or primarily relied on for their effectiveness, fully recognizing that most 
used some combination of these resources in practice (Anderson 1977; Hood 1986). 

A government regulation requiring a licence in order to use a particular pesticide, for exam- 
ple, is a policy tool expected to give effect to a set of policy objectives (in this case, a problem 
with externalities from pollution and information asymmetries between producers and consum- 
ers of sophisticated chemical products) within a set of aims (such as environmental protection 
and species preservation) and implementation preferences (such as market-based service delivery 
within a market mode of governance). Such a mechanism requires an organization to implement 
it, some funding to pay the personnel involved in that activity, information notices to regulatees 
that a licence is required and that the requirement will be enforced, and some legal authority to 
create a licence scheme and enforce it. Such an instrument thus involves the use of many types 
of governing resources, but the primary resource it relies on is the legal authority to enforce 
compliance, without which all the other resources would be ineffective and unnecessary. 

This taxonomy proved useful in providing a limited number of eight clearly differentiated 
categories of instruments (see Table 2.1). 


Contemporary Conceptions of Instrument Choice 


As Linder and Peters (1990: 307) noted, once the tools of government have been inventoried 
and classified, “the need to do something more becomes irresistible”. The next logical step was 
“to explore functional connections” involving “matching instruments to goals, policy problems, 
social impact and organizations”. Taking this next step towards the idea of policy design, as we 
have seen, requires clarifying the nature of the criteria by which experts assess policy tools and 
the nature of the contexts in which they can reasonably be anticipated to perform as expected. 

Kirschen et al., in their early 1964 work, had already gone some distance towards that goal 
by arguing that the key determinants of policy choice in the case of the economic instruments 
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they had identified were the economic objective or goal pursued and the structural and con- 
junctural context of the choice. The economic objectives, they argued, were determined by 
the interaction of political parties and their representatives in government, administrators, and 
interest groups (1964: 224—236) while the structural and conjunctural context, in turn, was 
affected by the influence of long-term economic processes and structures and current economic 
conditions (236-238). They argued that the actual choice of instrument from within the set that 
fit these epistemic and contextual constraints should be made on essentially technical grounds, 
according to efficiency and cost criteria, although the political preferences of interest groups 
and governments — including sociological and ideological constraints — and the institutional 
limitations of the political system itself had to be taken into account as factors influencing key 
decision-makers (238-244). 

This was a prescient analysis of the overall set of factors affecting instrument choices (Majone 
1976, 1989), combining as it did both technical and political factors. However, it was also one 
which was not adequately grounded in a classification of instruments so as to be able to pro- 
duce specific recommendations or hypotheses concerning appropriate instrument selections and 
policy designs in different circumstances or times. 

The first models of instrument choices which were so grounded attempted to identify a lim- 
ited number of criteria upon which policy tools varied, creating single or multiple “spectrums” 
or “continuums” of instrument characteristics which it was hoped could then be associated 
with specific government preferences among these criteria. Dahl and Lindblom, for example, 
as early as 1953 had argued that the number of alternative politico-economic instruments is 
virtually infinite and proposed five long continua as a method of assessing tool preferability in 
specific contextual situations (Dahl and Lindblom 1953). Their first continuum ranged instru- 
ments according to whether they involved public or private enterprises or agencies, the second 
according to whether they were persuasive or compulsory, the third according to whether they 
involved direct or indirect controls over expenditures, the fourth according to whether 
they involved organizations with voluntary or compulsory membership, and the fifth accord- 
ing to whether government agencies were autonomous or directly responsible to legislators 
or executive members. Although Dahl and Lindblom did not pursue any further the question 
of whether and to what extent governments actually used these criteria in order to choose a 
particular instrument, their idea of arranging instruments on a continuum in order to bet- 
ter clarify the reasons behind their choice was adopted by many authors. Salamon and Lund 
(1989), for example, suggested that different instruments involve varying degrees of effective- 
ness, efficiency, equity, legitimacy, and partisan support that affect their appropriateness for a 
particular situation. 

A simplified version of this model was put forward by a group of Canadian scholars includ- 
ing Bruce Doern, Richard Phidd, Seymour Wilson, and others, who published a series of 
articles and monographs in the late 1970s and early 1980s that turned Lowi’s two-dimensional 
matrix of policy choices into a single continuum of policy instruments based on the “degree of 
government coercion” each instrument choice entailed. They first placed only self-regulation, 
exhortation, subsidies, and regulation on this scale (Doern 1981) but later added in categories 
for “taxation” and public enterprise (Tupper and Doern 1981) and, finally, an entire series of 
finer “gradations” within each general category (Doern and Phidd 1983) (see Figure 4.3). 

For Doern and his colleagues, the development of their coercion spectrum model led to the 
hypothesis of a twofold rationale of instrument choice, one that fit very well with the notion 
of a “continuum” of choices and offered a great deal of explanatory power in the context of 
liberal-democratic states. This rationale was based on an appreciation of the ideological prefer- 
ences of liberal-democratic governments for limited state activity and the difficulties posed to 


25 


Michael Howlett 


this principle by the relative political “strength” of the societal actors in resisting government 
efforts to shape their behaviour. Assuming that all instruments were more or less technically 
“substitutable” — or could perform any task although not necessarily as easily or at the same 
cost — they argued that in a liberal-democratic society, governments, for ideological reasons, 
would prefer to use the least coercive instruments available and would only “move up the scale” 
of coercion as far as was necessary to overcome societal resistance to attaining their goal. As 
Doern and Wilson put it: 


[P]oliticians have a strong tendency to respond to policy issues, (any issue) by moving 
successively from the least coercive governing instrument to the most coercive. Thus 
they tend to respond first in the least coercive fashion by creating a study, or by creat- 
ing a new or reorganized unit of government, or merely by uttering a broad statement 
of intent. The next least coercive governing instrument would be to use a distributive 
spending approach in which the resources could be handed out to constituencies in 
such a way that the least attention is given as to which taxpayers’ pockets the resources 
are being drawn from. At the more coercive end of the continuum of governing 
instruments would be a larger redistributive programme, in which resources would be 
more visibly extracted from the more advantaged classes and redistributed to the less 
advantaged classes. Also at the more coercive end of the governing continuum would 
be direct regulation in which the sanctions or threat of sanctions would have to be 
directly applied. 

(Doern and Wilson 1974: 339) 


This model was lauded for its simplicity and elegance but, as critics pointed out, was still prob- 
lematical since “coercion” appears to be indivisible or, at best, still very difficult to operation- 
alize with the degree of precision required by the model (Trebilcock et al. 1982). Trebilcock 
et al. questioned the likelihood of state actors adhering to a minimalist notion of their own 
proper role in society, preferring public choice—inspired notions about bureaucratic expan- 
sionism and political credit mongering motivating administrative and political policymakers, 
especially notions of a political cost-benefit calculus aimed at vote maximization (Trebilcock 
et al. 1982). 

This formulation nevertheless has many advantages. It is not unidimensional, although it 
might appear so on first reading, because it, like Kirschen, does take into account several politi- 
cal and contextual variables, and it, like Cushman, assumes instrument choices are multi-level, 
with finer calibrations of instruments emerging after initial broad selections have been made. 
That is, it assumes that both states’ and societal interests in liberal-democratic regimes prefer 
a minimal state and choose instruments accordingly after that initial decision has been made. 
Preferring “self-regulation”, governments would first attempt to influence overall target group 
performance through exhortation and then add instruments only as required in order to com- 
pel recalcitrant societal actors to abide by its wishes, eventually culminating, if necessary, in the 
takeover of individual firms. 

This is not an unreasonable conclusion, based as it is on much observation of the practices of 
such governments, and hints at the “nested” nature of instrument choices, a subject not previ- 
ously as well developed in instrument studies. However, as Woodside (1986) argued: 


Experience suggest that governments do not always seek to avoid coercive solutions, 
but indeed, may at times seem to revel in taking a hard line from the start. While there 


are undoubtedly many reasons for these heavy-handed responses, surely some of the 
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most important ones include the constituency or group at which the policy is aimed, 
the circumstances in which the problem has appeared, and the nature of the problem 
involved. 


(786) 


These authors also questioned the notion of instrument substitutability found in Doern’s 
work, arguing that constitutional restraints, financial limitations, and other technical criteria pre- 
vented certain instruments from being utilized in specific circumstances (Trebilcock and Prich- 
ard 1983). Thompson and Stanbury did much the same, focusing on visibility and its linkages to 
political advantage and disadvantage as a criterion of instrument choice. Democratic politicians, 
they argued, are not ideologically predisposed towards a small state or minimal instruments but 
would adopt whatever instrument generates the most political benefits for them while minimiz- 
ing the political costs (Howard and Stanbury 1984; Stanbury 1986; Baxter-Moore 1987). 


Improving on Hood’s Taxonomy of Policy Instruments: Adding 
in Procedural Tools and “Non-implementation Tools” 


All this work, however, focused on implementation and the kinds of tools governments used to 
influence other actors in society to abide by their wishes. They generally did not examine other 
stages of the policy process — such as agenda-setting, policy formulation, decision-making, or 
policy evaluation — and also failed to examine the kinds of “procedural” tools governments have 
at their disposal in affecting these aspects of policy-making from the creation of new depart- 
ments and agencies to the spread of information, be it true or false. 

Most older literature on policy tools focused on single instrument choices and designs (Tupper 
and Doern 1981; Salamon 1989; Trebilcock and Prichard 1983), and these studies provide only 
limited insights into the complex arrangements of multiple policy instruments which are com- 
monly found in all policy fields (Jordan et al. 2011, 2012; Givoni 2013). Many significant issues 
related to the manner in which tool choices in bundles are made and how tool bundles evolve 
over time affect the propensity for designs to avoid the twin shoals of over- and underreacting 
to problems (Maor 2012; Howlett and Rayner 2007) while incorporating better knowledge of 
both synergistic and counterproductive tool relationships and interactions (Del Rio 2010; Lep- 
lay and Thoyer 2011; Grabosky 1995; Justen et al. 2013). For these authors, the key question was 
no longer “Why do policymakers utilize a certain instrument?” as it was for earlier generations 
of students of policy instrument choice, but rather “Why is a particular combination of proce- 
dural and substantive instruments utilized in a specific sector?” (Dunsire 1993; Howlett 2000; 
Salamon 2002; Cubbage et al. 2007; Gleirscher 2008; Gipperth 2008; Taylor 2008; Clark and 
Russell 2009; McGoldrick and Boonn 2010). Hence, for example, a well-known implementa- 
tion style found in many US policy sectors, dubbed “adversarial legalism” by Robert Kagan, is 
composed of a preferred substantive instrument — regulation — and a characteristic procedural 
one — judicial review — based on widespread, easily accessible legal procedures (Kagan 1991). 

Fortunately, though, earlier generations of implementation scholars had not completely 
neglected procedural instruments, and other works, like those of Gunningham and his col- 
leagues (Gunningham et al. 1998) had begun to deal with the problem of the design of policy 
mixes or combinations of procedural and substantive tools put in place to accomplish govern- 
ment ends. In the late 1990s, work on instrument selection began to assess the question of the 
potential to develop optimal policy mixes and to move away from a focus on single instru- 
ment choices (Grabosky 1994; Gunningham and Young 1997). Studies such as Gunningham, 
Grabosky, and Young’s work on “smart regulation” led to the development of efforts to identify 
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complementarities and conflicts within instrument mixes or tool “portfolios” involved in more 
complex and sophisticated policy designs (Barnett et al. 2009; Barnett and Shore 2009; Buck- 
man and Diesendorf 2010). 

Meanwhile, other scholars such as Salamon had also turned their attention to the issue of 
micro-calculations and calibrations of policy tools and developed ideas about the sets of factors 
policy formulators take into account in fine-tuning their instrument choices, such as deciding 
not just whether or not to provide a subsidy to industry, but how much to provide and in what 
fashion (Salamon 1989, 2002). 

And others, like Schneider and Ingram, developed more sophisticated notions of the inter- 
relationships existing between governance modes and target group behaviour which did not 
always assume that governments benevolently, or accurately, selected the correct policy tar- 
gets (Schneider and Ingram 1997). In their 1990 study of policy targets and their behaviour, 
Schneider and Ingram also began to systematically pursue Doern’s insight that the extent of 
a government’s willingness to alter the underlying behaviour of key policy actors was a major 
factor affecting its choice of policy implementation tools. They argued that policy-making 
“almost always attempts to get people to do things that they might not otherwise do” and 
noted that: 


If people are not taking actions needed to ameliorate social, economic or political 
problems, there are five reasons that can be addressed by policy: they may believe that 
law does not direct them or authorize them to take action; they may lack incentives 
or capacity to take the actions needed; they may disagree with the values implicit in 
the means or ends; or the situation may involve such high levels of uncertainty that the 
nature of the problem is not known, and it is unclear what people should do or how 
they might be motivated. 

(Schneider and Ingram 1990: 513—514) 


That is, they recognized that each of Hood’s “statecraft” resources required not only state capac- 
ity in that area — that is, a plentiful supply of the “resource” — but also a corresponding belief or 
endowment on the part of target groups which would allow that capacity to be utilized effec- 
tively (Schneider and Ingram 1990a, 1990b, 1993, 1994, 1997). 

Thus, the effective use of “nodality”, for example, requires the transmission of information 
to targets, “authority” requires the enforcement capability to coerce or force targets to do some- 
thing they might not otherwise wish to do, “treasure” requires having the fiscal capacity to pro- 
vide targets with incentives or disincentives to act in certain ways, and “organization” requires 
the administrative capacity to provide them with some good or service directly. But in order 
to be effective not only must governments have an adequate “supply” of these resources but 
targets must also be susceptible to their deployment: effective information transmission requires 
credibility or the belief among targets that a government is telling the truth; the effective use of 
authority requires legitimacy or the belief among the target population that the use of author- 
ity is legally and morally appropriate; the effective use of treasure resources requires cupidity or 
the willingness on the part of actors to accept payments or make them; and the effective use of 
organization requires trust on the part of the target group that an administration is competent 
and capable of actually delivering the promised goods or services. 

In the case of procedural instruments, several works dealing with aspects of the subject pro- 
vided a broad sense of which direction to pursue in attempting to elevate this area of instrument 
studies to the level that substantive implementation instrument research had attained through 
taxonomy construction and model building (Walker 1983, 1991; Qualter 1985). 
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In their 1988 work, for example, Bressers and Klok (1988) had noted the ways in which 
“subjective rational actors” can be influenced by manipulation of the alternatives placed before 
them and that different instruments can affect the number of policy options developed in the policy 
process or the calculations of the costs and benefits of alternative courses of action made by 
policy actors. While some of the instruments they examined were “substantive” (for example, 
the use of licences to affect the cost of certain activities), most of the instruments captured by 
their scheme were procedural, especially those dealing with the selective creation, provision, and 
diffusion of information to policy actors. 

On the basis of this analysis, Schneider and Ingram, following Elmore and his colleagues, 
identified five general types of instruments corresponding to these “behavioural assumptions”. 
These they called “authority”, “incentives”, “capacity-building”, “symbolic and hortatory” and 
“learning” instruments. As was the case with Bressers and Klok, this scheme included both 
“procedural” and “substantive” tools. While their discussion, like many in the USA at the time, 
virtually ignored pure public provision of goods and services by government agencies and cor- 
porations (Leman 1989), the “authority” and “incentive” examples cited are typical substantive 
instruments involving mixed provision of goods and services by a combination of private and 
public actors. “Capacity”, “symbolic”, and “learning” tools, however, are much more procedur- 
ally oriented, affecting the policy institutions and processes within which policy decisions take 
place. 

Taken together, the works of Bressers and Klok, along with those of Schneider and Ingram 
and others in the US and Europe, identified a large number of procedural instruments, their 
inventory, like that in the case of substantive tools, being accompanied by several ideas about 
how to classify them (Chapman 1973; Weiss and Tschirhart 1994). These authors identified, 
among others, tools involved in education, training, institution creation, the selective provision 
of information, formal evaluations, hearings, and institutional reform (Wraith and Lamb 1971; 
Chapman 1973; Kernaghan 1985; Peters 1992; Weiss and Tschirhart 1994; Bellehumeur 1997). 

Research into the tools and mechanisms used in intergovernmental regulatory design also 
identified several other such instruments, including intergovernmental “treaties” and a variety 
of “political agreements” that can affect target-group recognition of government intentions 
and vice versa (Bulmer 1993; Doern and Wilks 1998; Harrison 1999). Other research into 
interest-group behaviour and activities highlighted the existence of tools related to group cre- 
ation and manipulation, including the role played by private- or public-sector patrons in aiding 
the formation and activities of such groups (Burt 1990; Phillips 1991; Pal 1993; Finkle et al. 
1994; Nownes and Neeley 1996; Lowry 1999). Still other specialized research into aspects of 
contemporary policymaking highlighted the use of procedural techniques such as the provision 
of research funding for, and access to, investigative hearings and tribunals (Salter and Slaco 1981; 
Gormley 1989; Cairns 1990; Jenson 1994). 

While most researchers focused on the manner in which these instruments were used to 
enhance “desirable” traits in public policymaking, such as enhanced participation and the wider 
dissemination of policy-relevant knowledge, some scholars like Saward (1992) also emphasized 
that procedural tools were also used to “negatively” affect interest groups’ and other actors’ 
behaviour: that is, to restrict their freedom to associate and engage in policy-influencing activi- 
ties. This latter research highlighted the role such tools have played on the “dark side” of politics 
and policymaking: for example, suppressing government enemies and rewarding friends via 
punishment, exclusions, and denial of information (Goodin 1980; Saward 1990, 1992). 

Examples of “negative” procedural policy tools identified at this time included co-opting 
opponents through provision of funds and other privileges, denying information, keeping 
opponents’ views from the public, penalizing opponents by denying funding or recognition, 
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fragmenting opposition — divide and conquer — by selective rewarding, rewarding “neutrals”, 
adding administrative hurdles and costs to opponents, and many more. These latter studies, how- 
ever, all existed outside the mainstream of policy instrument research, which continued to focus 
almost exclusively on substantive implementation tools but were ready to be used when policy 
design and implementation studies moved in a procedural direction in the late 1990s. 

Hood’s taxonomy of substantive instruments can be modified to help make sense of this dis- 
parate list of procedural tools, and this task was undertaken in the late 1990s in several quarters. 
Classifying procedural instruments just as Hood had done for their substantive counterparts — 
that is, in accordance with the type of “governing resource” on which they primarily rely for 
their effectiveness — generates a useful preliminary taxonomy of procedural tools. 

Drawing a distinction between “positive” and “negative” uses of governing resources in terms 
of whether they encourage or discourage actor participation in policy processes further paral- 
lels the “effector-detector” distinction made in Hood’s original discussion of substantive tools 
(Howlett 2000) (see Table 2.2). 

As was the case with Hood’s discussion of substantive instruments, this taxonomy is useful 
insofar as it highlights a small number of different basic resources used by different types of proce- 
dural tools and therefore allows a virtually unlimited number of such instruments to be placed in 
a limited number of general categories, preparing the ground for the development of improved 
understandings of the basic contours and possibilities of tool selection and of policy designs. 

This insight allows a simplified NATO model to be set out in Table 2.3, which includes both 
procedural and substantive tools as well as a clearer idea of what constitutes a basic governing 
resource. It is this model which is used in many of the chapters which follow to set out and 
describe the basic subtypes and most common individual kinds of implementation instruments 
used in contemporary policy designs. 


‘Table 2.2 A Resource-Based Taxonomy of Procedural Policy Instruments 


Governing Resource 


Nodality Authority Treasure Organization 
Principle “Positive” Freedom of Mandated participatory Interest-group Conferences and 
Use information processes funding commissions 
“Negative” Propaganda Preferential access to Targeted campaign Red tape 
policymakers funding 


(Cells provide examples of instruments in each category.) 


Source: M. Howlett, M, “Managing the ‘Hollow State’: Procedural Policy Instruments and Modern 
Governance,” Canadian Public Administration 43, no. 4 (2000): 412-431 


Table 2.3 A Simplified Taxonomy of Substantive and Procedural Implementation Tools 


Governing Resource 


Information Authority Treasure Organization 
Purpose Substantive Public information Independent Subsidies and Public enterprises 
of tool campaigns regulatory agencies grants 
Procedural Official secrets acts Administrative Interest-group Government 
advisory committees funding reorganizations 
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Conclusion 


Policy tools, or the instruments and techniques used by government in order to implement 
policy goals (Howlett 2005), have a special place in considerations and studies of policymaking. 
This is because this approach is based on the preparation of plans for tool use with a reasonable 
chance of achieving a specific goal or target (Howlett 2004). Choosing policy tools becomes 
more complex when multiple goals and multiple policies are involved within the same sector 
and government, as is very common in many policy-making situations (Doremus 2003; Jordan 
et al. 2012; Howlett et al. 2009). 

These latter kinds of multi-policy, multi-goal and multi-instrument mixes — what Milkman 
et al. (2012) call “policy bundles”, Chapman (2003) and Hennicke (2004) a “policy mix” and 
Givoni et al. (2012) “policy packages” — are examples of complex portfolios of tools. These 
mixes typically involve much more than functional logics linking tools to a goal and also deal 
with ideological or even “aesthetic” preferences in tool choices and goal articulation, which 
involves trade-offs and bargaining between actors in choosing one set of tools, goals, and poli- 
cies over another (Williams and Balaz 1999). This makes their formulation or design especially 
problematic (Peters 2005; Givoni 2013; Givoni et al. 2012). Most often, the focus should move 
from the design of specific instruments to the appropriate design of instrument mixes. This is 
more difficult to do when instruments belong to different territorial/administrative levels. 

Key design questions about such portfolios with which contemporary scholars and practitio- 
ners grapple include the issues of avoiding both “over” and “under” design (Haynes and Li 1993; 
Maor 2012), how to achieve “complementarity” and avoid “redundancy” or counterproductive 
mixes (Grabosky 1995; Hou and Brewer 2010; Justen et al. 2013), how to enhance or alter mixes 
over time so that they continue to meet old goals and take on new ones (Heijden 2011; Kay 
2007), and how to sequence or phase in instruments over time (Taeihagh et al. 2013). 

As this discussion has shown, over the course of the past 30 years, the study of policy imple- 
mentation instruments has advanced through the various stages of social scientific theory con- 
struction and now contributes a great deal of knowledge to policy formulation and policy design 
(Hood 2007; Lascoumes and Le Gales 2007). 
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WHERE TOOLS ARE DEPLOYED 
IN THE POLICY PROCESS 


Policy Instruments and the Policy Cycle 


Michael Howlett 


The role played by implementation instruments in policy design has been a central focus of most 
research on policy tools. It is they which provide the substance or content of whatever design 
deliberations occur at the formulation stage. However, policy instruments exist at all the stages 
of public policymaking. There are specific instruments, like consultative mechanisms such as 
public hearings or investigatory commissions, which serve as agenda-setting instruments. And 
there are others, like the use of decision-making matrices and legislative committee systems, 
which are used at the decision-making stage of the process. Even further, auditor generals and 
other kinds of expenditure evaluation systems are instruments used at the evaluation stage of the 
policy cycle. How these tools operate and their effects on policymaking and policy outcomes 
are new areas of research in instrument studies. 


Introduction: Policy Tools and the Policy Cycle 


The exact processes by which policy decisions are taken vary greatly by jurisdiction and sector 
and reflect the great differences and nuances that exist between different forms of government — 
from military regimes to liberal democracies and within each type — as well as the particular con- 
figuration of issues, actors and problems various governments, of whatever type, face (Ingraham, 
1987; Howlett, Ramesh and Perl, 2009). 

In some circumstances, policy decisions will be more highly contingent and “irrational” — that 
is, driven by situational logics and opportunism rather than careful deliberation and assessment — 
than others (Kingdon, 1984; Cohen, March and Olsen, 1979; Dryzek, 1983; Eijlander, 2005; 
Franchino and Hoyland, 2009). This high level of contingency in decision-making has led some 
critics and observers of policy design efforts to suggest that policies are less “designed,” in the 
sense that a house or a piece of furniture can be, than arrived at through odd combinations of 
luck and timing (Dryzek and Ripley, 1988). However, many other scholars disagree with this 
assessment. They view policymaking not as primarily a random, ritualistic or symbolic form of 
state activity, but rather as a conscious matter of attempting to match the means of policy imple- 
mentation to policy goals, however those are formulated. That is, policymaking is viewed as an 
instrumental problem-solving activity, one in which various governing resources are marshaled 
into a set of techniques which could, at least potentially or theoretically, achieve the aims, objec- 
tives and goals of policymakers. 
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Either view, however, needs to grapple with the questions of where and when in the policy 
process policy tools are deployed. In this effort, it is helpful to envision the policy process as a 
series of stages or sub-stages in which policymakers grapple with problems and attempt to arrive 
at some sort of solution to them. A model of the policy process which is very helpful in this 
regard is that of the policy cycle (Howlett, Ramesh and Perl, 2020; Wegrich). 


The Policy Cycle Model 


The idea of viewing policy-making as a process or a cycle or activities has been used to model 
policy-making activities since the earliest days of the discipline (Hawkesworth, 1992) and is 
helpful when examining policy tools and the purposes to which they are put. 

A five-stage model of the policy process has been most commonly used (Lyden et al., 1968; 
Simmons et al., 1974; Brewer, 1974; Brewer and deLeon, 1983; Jones, 1984; Anderson, 1984) 
(see Figure 3.1). In this model, “agenda-setting” refers to the first stage in the process. This is 
the earliest stage in a sequence of policy development when a problem is initially sensed by 
policy actors and a variety of solutions put forward. “Policy formulation” refers to the second 
stage in the process in which specific policy options are developed within government. In this 
stage, the range of plausible choices is narrowed by excluding the infeasible ones, and efforts are 
made by various actors to have their favoured solution ranked highly among the remaining few. 
“Decision-making” refers to the third stage in which governments adopt a particular course of 
action or non-action. This can involve the adoption of none, one, or some combination of the 
solutions remaining at the end of the estimation stage. In the fourth stage of “policy implemen- 
tation,” governments put their decisions into effect. This involves the use of some combination 
of the tools of public administration in order to alter the distribution of goods and services in 
society in a way that is broadly compatible with the sentiments and values of affected parties. 
Finally, “policy evaluation” refers to the fifth stage in the process, in which the results of poli- 
cies are monitored by both state and societal actors, often leading to the reconceptualization of 
policy problems and solutions in the light of experiences encountered with the policy in ques- 
tion (Howlett, Ramesh and Perl, 2009). 


Stages of Applied Corresponding Stages in 
Problem Solving Policy Formation 


Problem Recognition Agenda Setting 
Proposal of Solution Policy Formulation 
Selection of Solution Decision-Making 


Putting Solution into 
Effect 


Policy Implementation 


Monitoring Results Policy Evaluation 


Figure 3.1 The Five Stages of the Policy Cycle and Their Relationship to Applied Problem Solving 
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5. Policy 1. Agenda- 
Evaluation Setting 
4. Policy 2. Policy 
Implementation Formulation 
Policy Cycle 
3. Decision- 
Making 


Figure 3.2 The Policy Cycle 


The stages in applied problem solving and the corresponding stages in the policy process are 
depicted in Figures 3.1 and 3.2. 

Policy instruments extend to activities located in all stages of the policy process, including not 
just policy formulation and implementation, where they have traditionally been studied, but also 
agenda-setting, decision-making and policy evaluation. 


Styles of Policy Activity in Each Stage of the Policy Cycle 


Agenda-Setting 


In the scholarly literature on agenda-setting, a useful distinction is drawn between the systemic 
or unofficial public agenda and the institutional or formal, official agenda. The systemic agenda 
“consists of all issues that are commonly perceived by members of the political community as 
meriting public attention and as involving matters within the legitimate jurisdiction of existing 
governmental authority” (Cobb and Elder, 1972). This is essentially a society’s agenda for discus- 
sion of public problems, such as crime or health care, water quality or wilderness preservation. 
The formal or institutional agenda, on the other hand, consists of only a limited number of 
issues or problems to which attention is devoted by policy elites (Baumgartner and Jones, 1991; 
Kingdon, 1984). Each society has literally hundreds of issues which some citizens find to be 
matters of concern and would have the government do something about. However, only a small 
proportion of the problems on the public or systemic agenda are actually taken up by policy 
actors actively involved in policy development. 


Policy Formulation 


Studies of policy formulation have emphasized the importance of actors interacting to develop 
and refine policy options for government (Freeman, 1955; Linder and Peters, 1990). But unlike 
agenda-setting, in which the public is often actively involved, in policy formulation the relevant 
policy actors are restricted to those who not only have an opinion on a subject but also have 
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some minimal level of knowledge of the subject area, allowing them to comment, at least hypo- 
thetically, on the feasibility of options put forward to resolve policy problems. 

Scholars over the years have developed a variety of taxonomies to help identify the key 
actors in these policy subsystems, what brings them together, how they interact and what effect 
their interaction has on policy development (Jordan, 1981, 1990a, 1990b; Jordan and Schubert, 
1992). Most of these distinguish between a larger set of actors with some knowledge of the 
policy issue in question and a smaller set in which actors not only have requisite knowledge but 
also have established patterns of more or less routine interactions with each other (Knoke, 1993). 

Membership in knowledge-based policy communities extends to actors such as state poli- 
cymakers (administrative, political and judicial), members of non-governmental organizations 
(NGOs) concerned with the subject, members of the media who report on the subject, aca- 
demics who follow or research the area and members of the general public who, for whatever 
reason, have taken an interest in the subject (Sabatier, 1987, 1988). In many issue areas, the pol- 
icy community also involves members of other organizations such as businesses, labour unions 
or various formalized interest groups or professional associations concerned with government 
actions in the sector concerned. In some cases, international actors such as multinational cor- 
porations, international governmental or non-governmental organizations or the governments 
of foreign states, can also be members of sectoral policy communities (Haas, 1992). A subset of 
these actors who interact within more formalized institutions and procedures of government 
are defined as members of policy networks (Coleman and Skogstad, 1990; Marin and Mayntz, 
1991; Pross, 1992). These policy networks include representatives from the community but are 
“inner circles” of actors who effectively hold the power to veto many policy options as unten- 
able or infeasible. 

The type and nature of options which come forward to governments from the policy formu- 
lation phase are affected by the interaction of networks and communities (Howlett and Rayner, 
1995; Smith, 1993; Smith, 1994; Howlett and Ramesh, 1998; Howlett, 2002a). In open sub- 
systems where networks have many members and communities share many idea sets, it can be 
expected that a propensity for new, radical alternatives to the status quo may be generated in 
the policy formulation process. In closed subsystems, where networks have few members and 
communities are dominated by a single idea set, on the other hand, a status quo orientation will 
emerge in the policy options developed and put before decision-makers. In subsystems where 
only a few actors make up the network but communities are open to new ideas, significant 
alternatives to the status quo may emerge from the formulation process, but usually over the 
opposition of network members. In subsystems where many actors deal with few ideas, marginal 
or incremental options tend to develop. 


Decision-Making 


Many early studies of policymaking in companies, governments and organizations conducted 
largely by students of public and business administration argued that decision-makers attempt to 
follow a systematic method for arriving at logical, efficient decisions. They argued that policy- 
makers achieved superior results when they first established a goal, explored alternative strategies 
for achieving it, attempted to predict its consequences and the likelihood of each occurring and 
then chose the option which maximized potential benefits at least cost or risk (Edwards, 1954; 
Gawthrop, 1971; Weiss, 1977; Carley, 1980; Cahill and Overman, 1990). 

This model was “rational” in the sense that it prescribed a standard set of procedures for poli- 
cymaking which were expected to lead in all circumstances to the choice of the most efficient 
means of achieving policy goals (Jennings, 1987; Torgerson, 1986). Pure “rational” models of 
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decision-making thought of policymakers as neutral “technicians” or “managers” who identify a 
problem and then find the most effective or efficient way of solving it (Elster, 1991). Many of the 
latest efforts to enhance the efficiency and effectiveness of public policy decision-making, the 
“evidence-based policy movement” (Pawson, 2006), focus on the application of a systemic eval- 
uative rationality to policy problems (Sanderson, 2006; Mintrom, 2007) in classic rational style. 

Empirical research into decision-making processes, however, has found a more complex 
world in which disputes and conflicts over the meaning and definition of policy goals and means 
were frequent and in which political processes of bargaining and negotiation often outweighed 
“rational” deliberations and calculations of costs and benefits. Policymakers were often found to 
be neither necessarily neutral nor competent, and other models of the public policy decision- 
making processes have argued this is not an accidental situation but rather an inherent and 
unavoidable characteristic of the policy-making exercise. 


Policy Implementation 


Implementation is the stage in the policy process in which public policy decision-making must 
be translated into action. It has long been considered one of the most difficult and critical stages 
in the policy process — the phase in which any deficiencies in the design of the policy or any 
vulnerabilities with respect to the external environment will become visible. 

The set of tasks practically associated with implementation is best understood as a “continuum 
of strategic and operational task functions” (see Table 3.1). Tasks related to implementation range 
from high-level “strategic” design considerations to more operational-level design and capacity- 
building tasks in later stages of the policy process. Strategic functions relate to the highest levels 
of policy formulation and overall responsibility for implementation and overlap considerably with 
other stages in the policy-making process, such as agenda-setting and decision-making. 

As important as these tasks are, implementation is often neglected in practice. Policymak- 
ers often fail to prepare the ground systematically for implementation, resulting occasionally in 
high-profile policy disasters and even more frequently in policies that perform far below expec- 
tation. One key reason for this neglect is the sheer complexity, both analytical and practical, that 
implementation poses. Another reason is implementation’s political sensitivity. In policy for- 
mulation and even decision-making, critical differences between stakeholders may be papered 
over by using vague language or even postponing decisions on mission-critical but politically 


Table 3.1 Implementation Tasks 


Policy Implementation Program Implementation Project Implementation 


(Emphasis on Strategic Tasks) <----------------------- > (Emphasis on Operating Tasks) 

e Constituency building e Program design e Clear objectives 

e Overall policy objective setting * Capacity building for e Defined roles and 
and design implementers responsibilities 

e Designing the implementation e Collaboration with multiple e Plans/schedules 
framework — overall groups and organizations e Rewards and sanctions 
responsibilities and resource e Expanding resources and e Feedback/adaptation 
allocations to different actors support mechanisms 

e Legitimization e Active leadership 


Resource mobilization 


Source: Adapted from Table 2.1 in Brinkerhoff and Crosby (2002), p. 25 
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or bureaucratically “sensitive” aspects of policies outright. This has the advantage of keeping a 
policy process moving forward and “buying time” for more supportive coalitions to be built. But 
the consequences of such avoidance become unavoidable during the implementation stage, in 
which coordinators will struggle to generate, allocate and control resources and interpret policy 
intentions. The intended outputs and results of a policy will fail to materialize even as negative 
side effects of policies become more evident. 

The high degree of interdependence among stakeholders involved in the integrated policy- 
making process increases the complexity and vulnerability of the implementation challenge. As 
a result, the stakes in “getting implementation right” — in designing interventions that make suc- 
cessful implementation more likely and in anticipating and building in mechanisms to overcome 
implementation difficulties — are particularly high calculations from the onset. 


Policy Evaluation 


The last stage of the cycle is policy evaluation. For many early observers, policy evaluation was 
expected to consist of assessing whether a public policy was achieving its stated objectives and, if 
not, what could be done to eliminate impediments to their attainment. Thus David Nachmias 
defined policy evaluation as “the objective, systematic, empirical examination of the effects 
ongoing policies and public programs have on their targets in terms of the goals they are meant 
to achieve” (Nachmias, 1979). 

Often, however, public policy goals are not stated clearly enough to find out if and to what 
extent they are being achieved, nor are they shared by all key policy actors. Moreover, the pos- 
sibilities for objective analysis are also limited because of the difficulties involved in the attempt 
to develop objective standards by which to evaluate government’s level of success in dealing with 
subjective claims and socially constructed problems. 

What is significant in the evaluative process is thus not so much ultimate success and failure 
but that policy actors and the organizations and institutions they represent can learn from the 
formal and informal evaluation of policies in which they are engaged. This can lead them to 
modify their positions in the direction of greater substantive or procedural policy change, or it 
can lead them to resist any alteration to the status quo (Majone, 1989). 

A significant variable in this regard is the capacity of an organization to absorb new informa- 
tion. As Cohen and Levinthal argued in the case of the private firm: 


[T]he ability to evaluate and utilize outside knowledge is largely a function of the level 
of prior related knowledge. At the most elemental level, this prior knowledge includes 
basic skills or even a shared language but may also include knowledge of the most 
recent scientific or technological developments in a given field. Thus, prior related 
knowledge confers an ability to recognize the value of new information, assimilate it, 
and apply it to (commercial) ends. These abilities collectively constitute what we call 
(a firm’s) “absorptive capacity.” 

(Cohen and Levinthal, 1990) 


Policy evaluations do not necessarily result in major policy change. That is, while the concept 
of evaluation suggests that an implicit “feedback loop” is an inherent part of the policy cycle, in 
many cases, this loop may not be operationalized (Pierson, 1993). Path dependence, in which 
policies are set on “trajectories” following some “critical juncture,’ can hinder policy change and 
learning (Pierson, 2000). Based on the idea of “increasing returns,” Pierson has argued that posi- 
tive feedback can support an existing policy without it necessarily being the best solution to a 
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problem (Pierson, 2000; Liebowitz and Margolis, 1995). Organizational-institutional properties 
(Eising, 2002; Olsen and Peters, 1996), including varieties of principle-agent models (Dunlop 
and Radaelli, 2010; Waterman and Meier, 1998), international governance regimes (Haas, 1992) 
and bargaining and mutual adjustment processes (Elgström and Jonsson, 2000) are also often seen 
as barriers to learning from policy evaluations. 


Studying Policy Tools 


As Chapter 1 has set out, policy tool use extends to both the means or mechanisms through 
which goals are given effect and to the goals themselves, since goal articulation inevitably involves 
considerations of feasibility, or what is practical or possible to achieve in given conjunctures or 
circumstances given the means at hand (Huitt, 1968, Majone, 1975; Ingraham, 1987). Even 
when the goals pursued are not laudable, such as personal enrichment or military adventurism, 
or when the knowledge or means utilized is less than scientific, such as religious or ideologically 
inspired dogma or implementation preferences, and even when these efforts are much more ad 
hoc and much less systematic than might be desired, as long as a desire for effective resource use 
in goal attainment guides policymaking, it will involve some effort at analysis. 

Policy alternatives are composed of different sets or combinations of the policy elements 
described here. Policy instruments are the techniques or means through which states’ attempt 
to attain their goals. Other terms have been developed in the field of policy studies to describe 
the same phenomena, such as “governing instruments,” “policy tools” and the “tools of govern- 
ment”, and while these sometimes are used to refer to different mechanisms and calibrations of 
policy means, they are more often used synonymously. 

These are the subject of deliberation and activity at all stages of the policy process as they 
affect both the agenda-setting and policy-formulation processes as well as being the subject of 
decision-making policy implementation and evaluation (Howlett, 2005; Howlett, Ramesh and 
Perl, 2009). 

They have a special place in the consideration and study of policy design because, taken 
together, they comprise the contents of the toolbox from which governments must choose in 
building or creating public policies. Policy design elevates the analysis and practice of policy 
instrument choice — specifically tools for policy implementation — to a central focus of study, 
making their understanding and analysis key design concerns (Salamon, 1981; Linder and Peters, 
1990). Instrument choice from this perspective is, in a sense, public policymaking, and under- 
standing and analyzing potential instrument choices involved in implementation activity are 
policy design. The role of a textbook in policy design thus is one of assisting “in constructing an 
inventory of potential public capabilities and resources that might be pertinent in any problem- 
solving situation” (Anderson, p. 122). 

As Figure 3.3 shows, although policy instruments appear in all stages of the policy process, 
those affecting the agenda-setting, formulation, decision-making and evaluation stages of the 
policy process are very significant and important in public management (Wu, Ramesh, Howlett 
and Fritzen, 2010) but understudied when compared to the very wide array of studies of imple- 
mentation tools and processes. 

This is because the key sets of policy instruments of concern to policy designers have 
historically been those linked to policy implementation in the first instance and to policy 
formulation in the second (Howlett, 2011). In the first category, we would find examples of 
many well-known governing tools, such as public enterprises and regulatory agencies, which 
are expected to alter or affect the delivery of goods and services to the public and government 
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Figure 3.3 An Example of the Range of Policy Instruments by Governance Mode and Stage of the Policy 
Cycle 


(Salamon, 2002), while in the second, we would find instruments such as regulatory impact 
or environmental impact appraisals, which are designed to alter and affect some aspect of the 
nature of policy deliberations and the consideration and assessment of alternatives (Turnpenny 
et al., 2009). 

The role played by implementation instruments in policy design has been a central focus 
of most research on policy tools. It is they which provide the substance or content of what- 
ever design deliberations occur at the formulation stage. However, as we have seen, policy 
instruments exist at all the stages of public policy-making. There are specific instruments, like 
consultative mechanisms such as public hearings or investigatory commissions, which serve 
as agenda-setting instruments. And there are others, like the use of decision-making matrices 
and legislative committee systems, which are used at the decision-making stage of the process. 
Even further, auditor generals and other kinds of expenditure evaluation systems are instru- 
ments used at the evaluation stage of the policy cycle (Wu et al., 2010; Heilman and Walsh, 
1992). 

How these tools operate and their effects on policymaking and policy outcomes are areas of 
research in instrument studies which other chapters in this book explore. 
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4 


THE NEED TO UNDERSTAND 
AND INTEGRATE PROCEDURAL 
AND SUBSTANTIVE TOOLS 


Achim Lang 


The chapter makes a clear distinction between substantive and procedural policy tools and dis- 
cusses various typologies and types of tools. The chapter stresses the need to integrate procedural 
and substantive tools in a comprehensive way in order to tackle issues such as policy integration 
and wicked problems. The emphasis is on procedural tools, which are of particular importance 
for boundary-spanning activities. Procedural tools are delineated according to different phases in 
the policy cycle, as well as according to the involved organizations (including their preferences 
and problem perceptions) and the quality of their interactions. 


Introduction 


Governments, like any other organization, have developed tools or instruments to advance their 
objectives. Policy tools represent a systematic description of “what government does” (Hood, 
1983) and what it intends to achieve. From this perspective, government can be conceptual- 
ized as a set of administrative tools to affect and supervise the behavior of individuals and 
organizations. 

The use of such tools dates back to prehistoric times when hunter-gatherer societies first 
developed a kind of functionally differentiated structure, and rudimentary planning and imple- 
mentation processes were developed. The study of policy tools can be traced back to prehistoric 
accounts 4,000 years ago during the Xia dynasty in China (Seeger, 1934), the philosophical 
treaty of the duties of a statesman (De Officiis) by Cicero, or the somewhat malicious recommen- 
dations of Machiavelli in The Prince. These practices and accounts rested on a narrow, authorita- 
tive, and hierarchical type of policy process based on charismatic and traditional leadership and 
domination (Weber, 2019). Even the legal-rational model of Western societies, developed over 
the last two centuries, still focused on authoritative (legal) means as the primary source of gov- 
ernment (Salamon, 2000). 

In the wake of the newly established policy sciences, a massive proliferation has occurred 
in studies analyzing tools of governments (Le Galés, 2011; Peters, 2000; Schneider & Ingram, 
1990). This has brought about a shift in research focus away from more descriptive and legal 
accounts of policy tools to the development of categories (Bemelmans-Videc et al., 2011; 
Hood, 1983; Howlett, 2020), the effects of tools on policy objectives (Lang, 2019; Schneider & 
Ingram, 1990), the choice of policy tools (Howlett, 2018), and configurational analyses of 


DOI: 10.4324/9781003163954-5 47 


Achim Lang 


policy mixes (Cejudo & Michel, 2021; Howlett, 2004). These new avenues of research mainly 
involve the study of substantive policy tools. However, the principal innovation in the study of 
policy tools came with a focus on procedural tools that reflected the need of governments to 
increase participation by interested parties in decision-making as well as the implementation 
of policy. This shift from government to governance was also reflected in the study of policy 
tools (Levi-Faur, 2012). 

This chapter has two overall aims. First, it provides a distinction between substantive and 
procedural tools and sketches the main characteristics of substantive policy tools. These charac- 
terizations are chosen because they are applicable to procedural instruments as well. Excellent 
summaries of substantive tools can also be found elsewhere (Hood & Margetts, 2007; Howlett, 
2018, 2020). Second, it takes stock of the main debates concerning procedural policy tools. 
Specifically, this chapter argues that studies of procedural tools take up the challenge of current 
political processes: namely, that participation by interested and affected parties is increasing, and 
the design of substantive tools is becoming increasingly complex due to the interdependencies 
of policy issues. 


Policy Tools 


Policy tools at the disposal of the government are “means used to address public problems” 
(Salamon, 2000) or to attain policy objectives (see also Bemelmans-Videc et al., 2011; Howlett, 
1991). These tools are embedded in a layered hierarchy involving “the overarching goals that 
guide policy in a particular field, the techniques or policy instruments used to attain those goals, 
and the precise settings of these instruments” (Hall, 1993). Hall gives examples of a policy aimed 
at alleviating the financial problems of the elderly — for example, an old-age pension — and its 
setting would be the level at which benefits were fixed (Hall, 1993). 

Others conceptualize policy instruments by the resources needed for them to be effective. 
Typical resources are authority, (financial) incentives, and information (often paraphrased as 
“carrots, sticks, and sermons”) (Bemelmans-Videc et al., 2011). Authority tools (sticks) coerce 
people into complying with a set of rules. Financial tools (carrots) or communicative tools (ser- 
mons) have a lower authoritative force and, as a result, leave greater scope for personal choice. 
A carrot tool motivates people to act in a certain way by (financially) encouraging or discourag- 
ing certain behavior. A communicative instrument (sermon) influences behavior by supplying 
information that induces people to act in a prescribed manner (Bouwma et al., 2018). Policy 
tools frequently come in different combinations or policy mixes and are applied in different 
contexts (Howlett et al., 2015). 

The following paragraphs provide an overview of the basic elements and categories of sub- 
stantive policy tools, which also serve to categorize procedural tools (Howlett 2000, 2020; 
Peters, 2000). 


Substantive Policy Tools 


Substantive policy tools are designed to modify the behavior of individuals or organizations. Les- 
ter M. Salamon argues that substantive tools are means that structure collective action in order 
to address a public problem (p. 1642). Such tools are packages of different elements, delivered as 
a type of good or activity (e.g., a cash or in-kind payment, a restriction or prohibition, or the 
provision of information) by a delivery vehicle for this good or activity (e.g., a direct provision of 
a service). The delivery vehicle is part of a larger delivery system including a set of organizations 
that are engaged in providing the good or activity (e.g., public, non-profit, or for-profit), which 
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are tied together by a set of rules, whether formal or informal, defining the relationships among 
the entities that comprise the delivery system (Salamon, 2000: 1643-1644). 

This encompassing classification by Salamon has been subject to various revisions, especially 
with regard to the type of good/activity. Indeed, most scholars differentiate these goods and 
activities regarding the underlying resources. The most popular typology of substantive policy 
tools was developed by Hood, who distinguishes tools for “detection” and tools for “effecting.” 
Detection tools or detectors are all tools used by governments to obtain information, while 
effecting tools or effectors are designed to have an impact on those outside government. The 
second distinction sums up the four basic underlying resources that governments have: “nodal- 
ity” (now generally referred to as “information”), “authority,” “treasure,” and “organization” 
(Hood, 1983). 

Nodality denotes, in the stricter sense of Hood, the property of being at the center of a star- 
like network, dominating and steering the flow of information (Hood, 1983). Most scholars 
have broadened this concept and argue that governments not only channel information but 
also produce and disseminate information. Consequently, information tools refer to any activity 
related to disseminating and channeling information (Howlett, 2020). 

Authority means the possession of legal or official power; authority tools are prescriptions 
authorized by the legitimate public body that permits, prohibits, or requires a specific behav- 
ior of individuals or organizations under designated circumstances (Hood & Margetts, 2007; 
Schneider & Ingram, 1988). 

Treasure denotes the possession of tradeable or transferable resources — financial, human, and 
permissions. Treasure gives governments the ability to exchange these resources to influence the 
behavior of citizens or organizations and to buy information (Hood & Margetts, 2007). 

Organization means the possession of competencies and skills (by people) and infrastructure 
(e.g., buildings, materials, and computers), giving governments the ability to act directly, using 
their own personnel rather than third parties (Hood & Margetts, 2007). In other words, organi- 
zation denotes the delivery system pertaining exclusively to public authorities. 

This typology is the starting point for many analyses pointing to the choice of instruments 
(Borras & Edquist, 2013; Capano et al., 2020; Howlett, 2018), instrument effectiveness (Bali 
et al., 2019; Schneider & Ingram, 1990), and for conceptualizing procedural tools (Howlett, 
2018, 2020). 


Procedural Policy Tools 


Effective public authorities must be sufficiently able to influence the stakeholders and their 
interactions within the decision-making and implementation processes. Such capabilities rest 
on the ability to deploy policy tools based on authority, information, organization, and trea- 
sure so that stakeholders engage in the policy process in a way they might not otherwise do 
(Schneider & Ingram, 1990, p. 513). Although numerous classifications of policy tools exist 
(Bemelmans-Videc et al., 2011; Hood, 1983; Howlett, 2000, 2005), procedural policy tools 
have received little attention (Howlett, 2000, 2005). Substantive policy tools are typically 
outputs of the policy-making process to resolve public concerns or problems. The process 
that involves the making and designing of tools concerns the management of policy networks 
since policymaking is increasingly less the sole responsibility of public authorities but involves a 
multitude of other organizations — governmental as well as commercial and not for profit. The 
same holds for the implementation phase when substantive policy tools have been defined and 
must be deployed. The central question is, therefore, how governing organizations can steer or 
influence policy networks in the policy-making and implementation phase. 
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Procedural policy tools “are used to manipulate the number or the nature of actors arrayed 
in the policy subsystems that policy-makers face” (Howlett, 2000, p. 420). Consequently, these 
tools are used to manipulate the policy-making behavior of stakeholders or policy-making pro- 
cesses, and public authorities usually have a range of tools at their disposal (Howlett & Ramesh, 
1998). Public authorities can choose from a set of procedural policy tools to establish a new 
setup that defines the boundaries and working conditions of the policy subsystem (Heclo, 1978; 
Sabatier & Weible, 2019) at different phases of the policy process (Ling, 2002). Governments 
may establish or alter networks or partnerships and restructure internal procedures to change 
policy choices and their design (Howlett, 2018). Examples of procedural policy tools include 
creating advisory committees to aid policy deliberations and decision-making in contentious 
areas (Howlett, 2000; Jochim & May, 2010; Klijn & Koppenjan, 2000). In addition, internal 
reorganization as a policy tool may affect departmental policy processes, such as other responsi- 
bilities or guidelines regarding the participation of lobbies or other political groups and differ- 
ent intradepartmental coordination procedures, fostering new operating arrangements affecting 
policy formulation (Howlett, 2020). 

In contrast to substantive policy tools, classifications of procedural tools are rare. Howlett 
(2000) adapts the classification elaborated by Christopher Hood (1983) to the study of proce- 
dural tools. In Howlett’s case, tools are grouped according to the resources on which they rely. 

Authority tools are the most common tools used by governments to achieve their pol- 
icy objectives, and this also applies to procedural policy tools. Using authority tools involves 
granting permission, prohibiting, or requiring action regarding the participation of individuals 
and organizations during policy formulation or implementation (Schneider & Ingram, 1990). 
Authority tools delineate the rules, establishing the number of people and organizations partici- 
pating in the policy networks, influencing their preferences and problem perceptions. Authority 
tools also provide rules for the decision-making process, establish a quorum for internal voting, 
and provide “softer” interaction rules such as a joint declaration of intent. Compliance mecha- 
nisms, in contrast, seek to guarantee participation in the more arduous implementation process. 
In addition, the accountability of political output and outcome serves to redefine stakeholder 
interests. Policy actors may be accountable to the coordination process or the quality of the 
policy output (Koliba et al., 2011; Ling, 2002). 

Information is another vital policy tool resource — and information tools include the provi- 
sion, suppression, and manipulation of information (Howlett, 2000). As Hood (1983) points 
out, information tools involve not only the management of information but also the positioning 
in a social network of information flows. Regarding the management of policy networks, mis- 
sion statements are expected to guide individuals’ preferences and perceptions. Monitoring and 
evaluation are also information tools (Howlett, 2000) frequently designed to boost commitment 
to the decision-making or implementation process by altering existing interests and making 
affected parties more amenable to alternative policy options (Kavanagh & Richards, 2001; Ling, 
2002). 

Treasure is the third important resource for governments on which procedural policy 
tools rest. Treasure is any material or immaterial medium (or money) with the capacity to 
be freely exchanged (Hood, 1983). Funding network activities and participation is impor- 
tant in network management included in this category (Howlett, 2000; Klijn et al., 1995). 
In addition, incentive-setting policy tools provide a tighter coupling of participants for the 
process (Ling, 2002). Neo-corporatist reasoning provides another fungible good, which can 
be included in the treasure category. In neo-corporatist arrangements, governments typically 
trade influence for compromise and commitment within the agreed objectives (Streeck & 
Schmitter, 1985). 
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Organization (Hood & Margetts, 2007; Howlett, 2000) or capacity-building (Schneider & 
Ingram, 1988) is another resource characterized by procedural tools. Organization denotes an 
institutional format in which interaction between stakeholders takes place alongside the institu- 
tion or organization (people, skills, and infrastructure) to shape and structure the interaction. 
These tools include all means designed to establish a political opportunity structure in which 
an institutionalized exchange occurs. Policy network analysis has revealed that these institu- 
tional venues channel information flows, making interaction more likely, even between oppos- 
ing groups (Howlett, 2000, 2005). Examples of such tools of organization include running an 
institutional venue such as an advisory board where the exchange takes place or an administra- 
tive unit, which Provan and Kenis (2008) call a network administrative organization (NAO) 
(Provan & Kenis, 2008). An NAO is set up expressly to manage and provide leadership activities 
to enable network members to accomplish their objectives. The NAO is not part of the network 
per se, meaning it does not provide public services and is not engaged in any planning activities. 

This overview of procedural policy tools demonstrates the variety of tool types. The follow- 
ing paragraphs summarize key research areas in which procedural policy tools are essential to 
the underlying theoretical framework. These paragraphs deal with the use of procedural tools 
at different phases of the policy cycle, establishing policy integration and coping with complex 
(“wicked”) problems. 


Procedural Policy Tools and Phases of the Policy Cycle 


Policy sciences and policy network literature provide an abundance of studies regarding inter- 
ventions in the organization and processes of the policy cycle (Jann & Wegrich, 2017; 
Lindquist & Wellstead, 2021). However, few studies have established a link with policy tools 
literature (Howlett & Mukherjee, 2017; Jordan & Turnpenny, 2015; Turnpenny et al., 2015). 
The following section provides a preliminary classification of procedural tools and their use in 
different phases of the policy cycle. The policy formulation and decision-making phase focuses 
on those processes that lead to the formally responsible decision-making institutions’ selection 
and adoption of a particular program or policy. The focus is on questions concerning interest 
negotiation and consensus building, and it is rarely the formal decision-making bodies alone that 
decide on a course of action. Most decisions are preceded by an informal exchange and nego- 
tiation process in various policy networks. Power resources, influence, and the distribution of 
competencies and responsibilities among the individual players involved (or the convergence of 
interests) determine the negotiation process and thus the shaping of policy content. Few studies 
have addressed policy tools for policy formulation, although Jordan and Turnpenny (2015) are 
a notable exception (Jordan & Turnpenny, 2015). Conversely, the implementation phase has 
received the most attention by policy tool scholars (Ptilzl & Treib, 2017). This phase focuses on 
how inputs come about and how they are processed into programs of interest and political out- 
put. In policy formulation and implementation, procedural tools are applied for the widespread 
involvement of stakeholders and the top-down alignment of their needs. Additionally, within 
the implementation phase, tools are related to the way specified outcomes of the collaboration 
process are effectively attained. 

Another distinction can be made in terms of network components. These involve the num- 
ber and types of involved organizations (including their preferences and problem perceptions) 
and the number and quality of interactions between organizations (Blanco et al., 2011; Klijn 
et al., 1995). Procedural policy tools, it is often argued, manipulate the number and configura- 
tion of links and the nodes of the policy or issue networks involved in policy formulation and 
implementation (Howlett, 2000; Kickert et al., 1997; Klijn & Koppenjan, 2000). 
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Table 4.1 summarizes the different policy tools in two-dimensional category space. The first 
quadrant introduces procedural policy tools that affect actor composition regarding goal consen- 
sus in the decision-making process. 

Policy formulation is dominated by informational tools, which are defined as “a technique, 
scheme, device or operation . . . which can be used to collect, condense and make sense of dif- 
ferent kinds of policy relevant knowledge to perform some or all of the various inter-linked tasks 
of policy formulation” (Jenkins-Smith, 1990, p. 11). 

Turnpenny et al. (2015) differentiate between tools for problem characterization (e.g., statis- 
tical indicators, surveys, and expert evidence), the specification of objectives (e.g., scenario anal- 
ysis), and the options assessment (e.g., cost-benefit analysis). These tools are designed to affect 
actor preferences and introduce an evidence perspective in the decision-making process. Tools 
affecting policy formulation are not restricted to informational tools but involve the invitation 
and funding of selected participants to join the formulation process and the exchange of influ- 
ence, compromise, and commitment to attaining this objective (Howlett, 2000). Authority tools 
focusing on the interaction between participants include the rules of the game and interaction 
guidelines specifying acceptable conduct. The provision and acknowledgement of various inter- 
est positions offer another informational perspective on policy tools. At the same time, establish- 
ing an appropriate opportunity structure in which participants may exchange information and 
other resources falls within the organization and treasure category of policy tools (Lang, 2019). 

Policy tools influencing the implementation phase contain compliance mechanisms that 
bind participants to the agreed objective and decisions. An example of such a tool is the neo- 
corporatist exchange of participation in the formulation process with support to implement 
the agreed policy (Streeck & Schmitter, 1985). In addition, reporting and monitoring tools are 
included whose aim is to raise the level of commitment to delivering results (Ling, 2002). While 
the link between implementation research and policy tools analysis is strongly top down, studies 


Table 4.1 Classification of Tools 


Process Network Structure 
Actors Jetwork Interaction 
Policy Actors and preferences setup Deliberation and consensus orientation 
Formulation Restrict/invite participation Rules for joint decision-making and 
Problem characterization tools co-design 
Elaboration of visions and scenarios Interaction guidelines 
Funding of participants Providing and acknowledging interest 
Options assessment positions 
Exchange of participation and Establishing political opportunity structures 
compromise (sponsoring of events/administration) 
Policy Commitment to delivering results Joint production orientation 
Implementation Trading participation in the Rules for joint production and collaboration 
formulation for commitment in Promotion of participation 
the implementation Joint action plans 
Delegation of implementation Incentives to collaborate 
competencies Integrating leadership 
Reporting and monitoring Setup of living/innovation labs 


Funding of participants 


Sources: Hood (1983), Lang (2019), Streeck and Schmitter (1985) 
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of co-production introduce a more bottom-up component (Howlett et al., 2017; Nesti, 2017; 
Sorrentino et al., 2018). 

Establishing rules for joint production can enhance implementation — that is, the interactions 
that lead to the desired outcomes. Incentives for collaboration are aimed at increasing the will- 
ingness of participants to take part in joint action. The setting up of public living or innovation 
labs (Nesti, 2017), whereby public authorities and other organizations or individuals innovative 
public services, is highly relevant in this context. This is because these labs involve not only 
infrastructure, venues, and financial resources but also soft tools such as integrating leadership 
(Bussu & Tullia Galanti, 2018). 

This short enumeration and classification of procedural tools is far from exhaustive but high- 
lights the wealth of tools available for each policy phase. 


Procedural Policy Tools for Policy Integration 


The literature of the policy cycle sees policy formulation and implementation entrenched in a 
steady policy subsystem in which domain-specific experts incrementally refine policies (Heclo, 
1978; Sabatier & Weible, 2019). However, an increasing number of policy issues demand inter- 
sectoral coordination to find adequate solutions. Therefore, policy integration and related theo- 
retical approaches examine cooperation between players from two or more policy domains in 
transferring policy objectives and tools derived from one policy domain to another (Tosun & 
Lang, 2017). This research perspective primarily refers to the difficulties that can arise when 
formulating and implementing cross-sectoral policies (Jordan & Lenschow, 2010). 

The literature on policy integration has developed a comprehensive collection of procedural 
policy tools. Studies focusing on boundary-spanning institutions suggest that integrating institu- 
tions are conducive to policy integration because they help focus the attention of those involved 
in the formulation process (Jochim & May, 2010) subsume parliamentary committees and gov- 
ernment-related advisory bodies. While information exchange and bargaining are the dominant 
modes of interaction, inter-administrative coordination provides a more active use of procedural 
tools by the government. Various tools are mentioned in the literature (Cejudo & Michel, 2021; 
Ling, 2002; Peters, 1998): new types of organizations that require flexible organizational struc- 
tures, superministries that internally coordinate cross-cutting issues, advisory committees and 
boards consisting of experts and laypersons, coordination organizations, task forces and working 
groups for interdepartmental coordination, and ad-hoc committees of affected organizations — 
new types of collaboration between organizations focusing on shared processes and structures, 
shared budgets, and shared teams. 

In a recent conceptual article, Cejudo and Michel (2021) propose a classification of policy 
integration tools: policy frame, authority, and information. A policy frame denotes a common 
understanding of the problem to be tackled and the responsibilities each participating organiza- 
tion was assigned in attaining a shared goal. Authority is defined by Cejudo and Michel (2021) 
as the “capacity that the decision-maker has to carry out the changes required to keep the policy 
mix pertinent, relevant and functioning as planned over time” (Cejudo & Michel, 2021, p. 4). 
Finally, information involves the strategic creation, collection, and utilization of data regarding 
the attributes of the tools within the procedural policy mix (Cejudo & Michel, 2021). In par- 
ticular, the policy frame seems to be a valuable addition to the orthodox classification of tools. 

Policy integration approaches aim to describe policy tools and consider various explanatory 
factors for the successful integration of cross-sectoral policies. For example, concerning the 
integration of climate change policies, Nilsson and Nilsson (2005) argue that they can only be 
integrated with measures from adjacent policy fields if the underlying policy problem can be 
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presented in such a way that all sector-specific actors feel affected by it and responsible for solving 
the problem. Jordan and Lenschow (2010) identify other potential success factors for policy inte- 
gration in their literature review. These include institutional factors such as the degree of minis- 
terial departmental independence and the design of federal-state structures. According to Jordan 
and Lenschow (2010), a legalistic administrative culture also tends to be an obstacle to successful 
policy integration. Moreover, they argue that constellations of interests impact the willingness 
to cooperate and integrate into the context of policy integration. Other studies emphasize the 
importance of trust between organizations and individuals for successful inter-administrative 
collaboration (O’Flynn et al., 2011). Thus, organizational change needs to be accompanied by 
cultural measures to foster a collaborative culture oriented toward inter-organizational problems 
(O’Flynn et al., 2011). In this context, Davies (2009) argues that conflicts of interest between 
administrative units, in particular, can prevent successful cooperation. In this case, the creation 
of special incentives ought to increase the willingness to cooperate. Studies dealing with inter- 
administrative coordination also highlight the positive effects of performance and output metrics 
(Kavanagh & Richards, 2001; Ling, 2002). In addition, it is argued that differences in problem 
perceptions and diverging interest reduce the stability of policy integration in the absence of 
integrating institutions (Jochim & May, 2010). 


Procedural Policy Tools and Wicked Problems 


Recently, analysis pointing to the significance of complex or “wicked” policy problems has 
emerged as a new field in the study of policy tools (Alford & Head, 2017; Head, 2019; Peters, 
2017). Wicked problems generally involve different layers associated with interest pluralism 
(multiple and diverging interests and actor values), institutional complexity (multiple venues of 
inter-organizational cooperation and governance), and knowledge uncertainty (fragmentation 
and gaps in reliable knowledge) (Head & Alford, 2015). The policy problem for government 
is how to address these when there are no ready-made and agreed solutions to hand. When 
governments have to consider multiple and sometimes conflicting goals, maximizing each goal 
becomes difficult, if not impossible (McConnell, 2018). Procedural tools are essential for tack- 
ling wicked problems since interest pluralism and knowledge uncertainty require integrated 
responses involving many players. As Howlett (2009) argues, these integrated responses require 
the identification and scoping of the problem, the deliberation on the choice of tools and pro- 
cedures, and the participation of many players in the implementation of solutions. However, the 
analysis of complex issues has shown that the government responses to wicked policy problems 
are often to fragment the issues into more manageable elements (Head, 2019). Consequently, 
one of the core challenges in policy design is selecting policy tools or a mix of tools appropri- 
ate to the task. Notwithstanding, tool selection remains a significant challenge for governments 
since impacts and the likely effects of various policy tools are difficult to assess, especially ex-ante 
(Head, 2019; Head & Alford, 2015). 

McConnell (2018) points out that the political and institutional dimensions of tool choice are 
just as crucial. Policy design and implementation involve negotiation, persuasion, and dealing 
with difficult trade-offs in finding and constantly adjusting substantive policy tools (McCon- 
nell, 2018). Kirschke and Kosow (2021) analyze the design of policy tools for emerging wicked 
problems and conclude that both comprehensiveness and diversity are important when coping 
with the complexity of wicked problems. They argue that multiple entry points are needed to 
address multi-faceted issues. In highlighting the heterogeneity of the target groups, they contend 
for an equally specific design of procedural policy tools (Kirschke & Kosow, 2021). Similarly, 
Endl (2017) argues in his analysis of wicked problems in the realm of sustainable development 
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that policymaking should take place across multiple levels or hierarchies as well as at political- 
administrative and territorial levels. Participation is a prerequisite for integrating different inter- 
ests, perspectives, and knowledge bases into the decision-making process. A key point to address 
is a long-term vision that unites all the various players in a shared vision of the end solution. 
Continuous adaptive learning is another pillar of this response (Endl, 2017). 

These empirical studies conceptualize procedural policy tool response to wicked problems 
as a unique and more complex case of policy integration. However, the “problem complexity” 
and “stakeholder divergence” elements of wicked problems are particularly challenging to tackle 
and by far outweigh any issues of interdepartmental coordination, mutual adjustment, and inte- 
grative stakeholder participation dominant in policy integration approaches. Head and Alford 
(2015) suggest developing and adopting “new approaches from systems thinking, collaboration 
and coordination, together with an adaptive-leadership and learning approach by public leaders 
and managers” (Head and Alford, 2015, p. 711). These approaches come close to the literature 
of network management (Edelenbos et al., 2013; Klijn & Koppenjan, 2014) and complex adap- 
tive systems (Teisman et al., 2009). However, this new thinking has yet to be elaborated further. 


Conclusion and Future Research Directions 


The study of substantive and procedural policy tools has advanced considerably in the last 20 years, 
but the development happened at different speeds. Substantive and procedural policy tools are rel- 
evant at different phases of the policy cycle. Substantive tools are typically the output of the policy 
process, which, in turn, is affected by the use of procedural tools. Since substantive tools directly 
affect the behavior of citizens and organizations, they have received the bulk of the attention. 

There exist several useful typologies for substantive tools while procedural tools are still 
underspecified. The literature of substantive tools has advanced to explaining the choice of tools, 
the effects and mixes of tools, and the specification of other contextual factors such as the policy 
domain in which substantive tools are applied. In contrast, scholars in procedural tools still 
struggle to develop typologies other than the ones based on resources. Typologies may rest on 
the different policy phases, the number and degree of integrating of policy subsystems, as well as 
the wickedness of the problem that needs to be tackled. However, developing useful typologies 
and classifications is the first step to advance to more sophisticated research design that includes 
factors affecting the choice of tools and their effectiveness. Studies in substantive tools have gone 
that way; scholars of procedural tools need to catch up. 
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5 
HOW TOOLS WORK 


Policy Instruments as Activators and Mechanisms 


Giliberto Capano and Michael Howlett 


The mechanistic approach to policy tool research tries to unpack the processes through which 
instruments reach (or do not reach) their expected goals by shedding light on the set of behav- 
ioural mechanisms and how the deployment of specific kinds of policy tools activates some com- 


pliance responses and not others in policy targets or ‘policy takers’. The mechanistic perspective | 
assumes that instruments are not simply independent variables or co-variables; rather, they are 
‘real drivers of specific outcomes of interest. As a result, policy instruments are considered to be 


‘activators’ of specific mechanistic chains through which the behaviours of individuals, groups 
and subsystems are altered in order to achieve specific outcomes. This means that instruments 


should not be considered as having a direct impact on the expected outcome, but rather serve as 
‘triggers’ capable of activating a sequence of individual and aggregate behaviours that is expected 
to achieve a specific result. In general, each category of tool involves the use of a specific gov- 
erning resource, but this resource use is expected to trigger or lever a specific characteristic 
or receptor in targets, inducing a certain behavioural response. Thus, the effectiveness of the 
deployment of such tools is linked not just to resource availability — a precondition of their use — 
but also to the existence of different ‘receptors’ on the part of policy targets which make them 
respond in a predictable way to the use of this resource when deployed. The policy mixes or 
bundles of policy instruments which arise in such cases typically involve not only a number of 
tools but also a range of motivations across a range of targets. This makes the assessment of the 
motivational structure of a policy realm more complex and difficult. It also suggests that rather 
than think about compliance in the context of single target-single instrument dynamics, policy 
design should centre on multiple target-multiple instrument ones. 


Introduction: Instruments as Activators 


What happens when a policy tools is deployed? Much existing policy thinking jumps from a 
proposed solution to an outcome, bypassing the “black box’ of behavioural and organizational 
change and targeting behaviour which generate outputs which allow that outcome to occur 
(Astbury and Leeuw 2010). 

A mechanistic approach to policy tool use helps make sense out of what it is, in general, that a 
policy tool does. Such a mechanistic perspective not only reinforces existing analyses and expla- 
nations of how design works, but it also shows how the policy capacity of government can be 
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strengthened by better understanding how policy tools ‘work’. That is, a mechanistic approach 
strengthens decision-makers’ analytical capacity by making it clearer what should be analyzed 
and why when it comes to choosing and deploying a policy instrument. This approach better 
allows appropriate policy tools to be chosen to ‘fit the job’ and also helps inform the calibration 
of those tools to ensure their effectiveness. 


A Mechanistic Perspective on Policy Instruments 


Over the past few decades, many social sciences have moved in a mechanistic direction as a 
result of dissatisfaction with both legalistic and statistical explanations. Starting with Elster (1989; 
Elster 2009), studies have sought to apply mechanistic explanatory logic to both political science 
and sociology, for example, including Hedström and Swedberg 1998; Tilly 2001; Mayntz 2004; 
Schmidt 2006; Abbott 2007; Hedstrom 2005, 2008; Gross 2009; Mahoney 2001; Gerring 2007. 

A mechanism is a ‘causal structure that explains the empirical outcome’ (Bygstad et al. 2016, 
83). These mechanisms differ from ‘intervening’ variables because they are part of a differ- 
ent perception of causation than the correlation logic commonly found in the field which 
defines ‘dependent’, ‘independent’, and ‘intervening’ variables (Hedström and Swedberg 1998). 
Although in general difficult to observe directly, they indicate precisely how X actually produces 
Y under specific conditions rather than simply chronicling the co-appearance of Y whenever 
X is present. 

Inspired by Coleman (1990) and Hedström and Swedberg (1998), a macro-meso-micro 
dynamic can be distinguished in three general types of mechanisms at work in the social and 
policy world (see Figure 5.1): ‘situational mechanisms which (on the basis of existing structural 
and environmental forces) constrain individuals’ actions or shape their beliefs, ‘action-formation’ 
mechanisms which link individual activities or behaviour to individuals’ actions and ‘transfor- 
mational’ mechanisms which generate intended and unintended outcomes (meso level). The 
persistence or institutionalization of the outcomes of transformative mechanisms then generates 
a new situation, in which these ‘first-order’ and other ‘second-order’ mechanisms such as learn- 
ing and feedback again occur. 

This view, common in sociology, however, assumes that there is no direct influence between 
the micro and the macro (emergent) levels (Bhaskar 2008) and is hence less useful in the 
policy realm than in others where structural or system-level relationships are also important. 
Rather than use this rubric, instead a mechanistic sequence for policymaking composed of 
three components — ‘activators’, ‘first-order mechanisms’ and ‘second-order mechanisms’ — is 
described next (see also: Capano et al. 2019; Capano and Howlett 2021). 


Activators, First- and Second-Order Mechanisms 
and Their Relationship to Policy Tools 


‘Activators’ are not mechanisms; rather, they are ‘events’ or activities which trigger mechanisms, 
activating the first-order and second-order mechanisms through which the behaviour of indi- 
viduals, groups and subsystems is altered in order to achieve a specific outcome. From a policy 
design perspective, activators are policy instruments through which decision-makers set up their 
policies to impact the reality they want to maintain or change. These instruments, of course, can 
be calibrated or tuned more precisely to attain government goals: for example, when a subsidy 
is adjusted upwards in the expectation that it will be utilized by more participants and enhance 
compliance rates with government initiatives. 
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First-order mechanisms are those which are triggered by the tool’s applications of state 
resources in order to affect the behaviour of individuals, groups and structures and can be applied 
in a specific fashion in order to achieve a specific outcome. In the policy realm, these mecha- 
nisms exist at the individual level, whereby actions like the provision of subsidies are expected to 
change individual savings behaviour; at the group level, whereby, for example, the provision of 
tax credits for charitable donations affect group membership and behaviour; and at the system 
or subsystem level, whereby adding or removing new actors and ideas, or reinforcing existing 
ones, through activities such as creating authoritative policy advisory boards and commissions, 
providing access to information and other kinds of tools affect system structure and behaviour. 
These different levels of mechanisms interact with each other as, for example, occurs when a 
tax-credit for charities changes individual behaviour, which can affect the group’s behaviour 
and, ultimately, the nature of a policy subsystem. 

Second-order mechanisms are those which are used to inform the use of activators by obser- 
vation of the reaction of individual, group and system behaviour to the previous deployment of 
activators. In the policy realm, these second order mechanisms are those which promote reflexive 
governance and include various kinds of activities such as policy learning, diffusion and transfer, 
both with respect to lessons learned about individual behaviour and collective or structural effects. 
They also include counter-causal mechanisms (counter-mobilization, negative framing, resistance) 
that can impede expected outcomes from first-order tools (Weaver 2010; Dunlop 2017). 

Figure 5.1 presents an elaboration of this arrangement. 


Activating First- and Second-Order Effects 


This model is useful in understanding what specific first- and second-order mechanisms can be 
triggered by the deployment of policy tools to affect actors’ behaviour in predictable directions 
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Figure 5.1 The Mechanistic Process From a Policy Design Perspective 
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and in what contexts. That is, first- and second-order mechanisms work in a context that varies 
according to the type of policy and can affect the ability of tools to alter behaviour in expected 
ways, depending on factors such as resource availability, capacity, implementation barriers and 
counteracting pulls on individual, group and subsystem activity and behaviour. 

Such a mechanistic perspective helps us understand the various dimensions of policy 
instrument use. The focus on ‘activators’, the design tools/choices through which an inter- 
vention operates, for example, helps distinguish between the content of the design in terms 
of adopted policy tools or the strategy for an intervention and the mechanisms that they are 
capable of activating. The focus on ‘first-order mechanisms’, in turn, requires an awareness 
of the types of mechanisms that are activated through policy design in order to address targets’ 
behaviour in such a way that their related behaviour produces the expected outcome. And 
the focus on second-order mechanisms helps us understand what occurs when a designed 
policy is implemented and thus improve the understanding of how a better policy design 
can be achieved. When a government decides to change how its education system functions, 
for example, if it has the requisite capacity to do so, it can activate first-order mechanisms, 
such as school competition or enhanced institutional accountability in order to improve 
the performance of schools, by activating second-order mechanisms like learning which 
is expected to do the same, or in some combination. Introducing vouchers or a combina- 
tion of choice and national testing, for example, can activate competition and institutional 
accountability. 


Activating First-Order Mechanisms 


When a mechanistic perspective is applied to policymaking, the key questions are how and why 
certain tools activate specific mechanisms. As set out in Figure 5.2, the mechanisms approach to 
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Figure 5.2 The Behavioural Expectations of Policy Tools 
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policymaking and policy dynamics centers around the idea that the use of policy tools activates 
certain propensities on the part of policy actors, leading to policy outputs resulting from more 
or less predictable changes in target behaviour and, ultimately, policy outcomes. 

This is a process which involves complex causal chains centered around existing policy 
behaviours and policy-making contexts and policy interventions which trigger — intentionally, 
consciously or not — policy mechanisms which affecting ‘target’ behaviour, changing it in some 
new direction (Falleti and Lynch 2009; Hedstrom and Swedberg 1996, 1998; Hedström and 
Ylikoski 2010). 

The linkages between policy instrument invocation and behavioural or policy change are 
very rich. As Figure 5.3 shows, a mechanistic process of behavioural change involves at least four 
linkages, all of which are affected by contextual aspects present at the exact moment at which 
instruments are invoked and mechanisms triggered. These are: 


1. The link between tools and the governing resources present at any moment in time. 
The link between resources and the mechanisms which tools activate. 

3. The links between the mechanisms and the actual behavioural changes which occur 
post-activation. 

4. The link between changes in behaviour and changes in policy outputs and outcomes. 


This approach thus views policymaking as largely about affecting behavioural changes in 
target populations, with policy instruments used as a means to influence a shift from an existing 
behaviour (‘Behaviour 1’) to a reformed or new one (‘Behaviour 1A or 2’) (Balch 1980), a shift 
which is moderated by the context in which the tool is deployed and the manner in which the 
tool is calibrated. 

Context is important because all four of these linkages — instrument choices, mechanism 
activation, reception and impact — are susceptible to various barriers and impediments linked to 
factors such as the number and type of targets, the availability (or not) of adequate resources or 
capacities on the part of state actors and the like (Howlett and Rayner 2013a, 2013b). As Figure 5.3 
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shows, in general, each link in a mechanistic chain is affected by contextual factors which can 
serve to block or make the linkages across the tools-output chain problematic: that is, making a 
design outcome more difficult to predict and control (Falleti and Lynch 2009). 


Factors Affecting Activation 


There are many such barriers and factors, which include such factors as the preferred policy 
style and governance mode, which can affect preferences for certain tools over others and the 
various resource (‘nodality’, ‘authority’, ‘treasure’ and ‘organization’, or NATO) (Hood 1986) 
endowments enjoyed by a government which can limit its capability to use particular tools or 
how they are calibrated or eliminate them altogether (Wu et al. 2015). Possible countervailing 
demands and constraints on behavioural change which can undermine the effect and impact of 
a mechanisms on subsequent behavioural change (Weaver 2014, 2015; Howlett 2018) are also 
important as are various kinds of implementation and other issues which can lessen or enhance 
policy outputs (Lindqvist 2016; Hupe and Hill 2016). 

Moreover, different temporal dimensions also exist in these relationships, such as the tempo, 
duration, time frame and timing of interventions and responses (Adam 2004). Some mechanistic 
causal chains need time to develop an outcome (duration), and if the designer does not take this 
into account, they may design in a very ineffective way. And some mechanistic chains can have 
a different timing with respect to other related or linked dynamics such as a low synchronization 
rate between first-order mechanism activation and second-order learning or diffusion; this tem- 
poral misalignment can be the bearer of unexpected outcomes with respect to designers’ goals. 
In other words, mechanisms can be activated in different ways and induce expected behaviors in 
an intense way or more loosely and at a slower pace. 


Individual- and Group-Level Behavioural Mechanisms 


First-order mechanisms are those psychological and structural characteristics of policy actors 
which directly affect their behaviour and reaction to policy cues. Although a great deal of the 
extant policy literature deals with individual-level behaviour, these mechanisms exist not only at 
the ‘individual level but also at the ‘group’ and ‘structural’ ones (Falleti and Lynch 2008). 

Most of the literature on causal mechanisms in general, and dealing with policy mechanisms 
in particular, has focused on the individual level. At this level, both individual-level ‘micro’ 
mechanisms (affecting either ‘system 1’ unconscious or semi-conscious psychological propensi- 
ties) and ‘system 2’ conscious or more ‘rational’ ones compose a key set of mechanisms which 
many policy tools are expected to activate (Kahneman 2013). 

In this view, at the individual level, the mechanisms activated by policy instruments in order 
to trigger policy change are characteristics of human behaviour such as greed, fear, risk aver- 
sion or the use of heuristics and other less rational ways of thinking, which affect the logics of 
calculation and appropriateness individuals take towards such issues as whether or not to perform 
a crime or quit smoking or invest in a pension fund or donate to a charity (March and Olsen 
2004). 

These individual-levels mechanisms are triggered or activated by ‘substantive’ policy instru- 
ments (Howlett 2000), which are the typical kinds of policy tools discussed in the literature 
around economic incentives and disincentives such as the provision of subsidies or the creation 
of regulatory regimes (Tupper and Doern 1981; Hood 1986; Salamon 2002). As Hood noted, 
these tools rely on a set of governing resources for their effectiveness, including ‘nodality’ 
(or information), authority, treasure and the organizational resources of government (Hood 
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1986; Anderson 1975) (see Table 5.1) which form an important part of their capacity of induc- 
ing behavioural responses and figure prominently in thinking around their likely effectiveness 
on the ground. 

Thus, information-based instruments, for example, can facilitate the provision of informa- 
tion as well as suppress it and can involve the release of misleading as well as accurate information 
(Goodin 1980), both of which can affect human cognitive and emotional response mechanisms: 
for example, concerning whether and how many supplies to stockpile in the face of a natural 
disaster or threat. These tools can be calibrated or applied at different levels of intensity, affecting 
the degree or speed to which a mechanism is activated 

One of the main reasons such tools might be deployed is supply oriented: that is, a govern- 
ment may utilize specific kinds of tools which deploy the kinds of resources it has in ample sup- 
ply or which could be easily replenished (Hood 1983). But in addition to ‘supply-side’ capacity 
issues, ‘demand-side’ considerations are also very significant in such choices. That is, in general, 
each category of tool involves the use of a specific governing resource expected to trigger or 
lever a specific characteristic or receptor in a target, inducing a certain behavioural response. 
Thus, the effectiveness of the deployment of such tools is linked not just to resource availability — 
a precondition of their use — but also to the existence of different ‘receptors’ on the part of policy 
targets which make them respond in a predictable way to the use of this resource when deployed 
and to the level of knowledge policymakers have concerning those propensities. 

Table 5.2 presents a framework of the behavioural prerequisites which governing tools rely 
on for the effect. 


Table 5.1 A Resource-Based Taxonomy of Procedural and Substantive Policy Instruments 


Governing Resource and Target Need 


Information Authority Treasure Organization 
Purpose Substantive Public Information Independent Subsidies and Public Enterprises 
of Tool Campaign Regulatory Agencies Grants 
Procedural Official Secrets Administrative Advisory Interest Group Government 
Acts Committees Funding Reorganizations 


(Cells provide examples of instruments in each category.) 


Source: Adapted from Howlett (2000) 


Table 5.2 Behavioural Needs for Resource Effectiveness 


Tool Type Statecraft Resource Applied Target Behavioural Prerequisite 


Nodality Information Credibility/Trust — willingness to believe and act on 
information provided by government 

Authority Coercive Power/Force Legitimacy — willingness to be manipulated by government- 
invoked penalties and proscriptions 

Treasure Financial Cupidity — willingness to be manipulated by gain/losses 
imposed by governments 

Organization Organization Competence — willingness to receive goods and services 
from government and enter into partnership arrangements 


Source: Howlett (2011) 
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In the case of information use, for example, tool effectiveness relies on both the availability of 
knowledge and reliable data and the means to distribute it (‘resources’) and the target’s belief 
in the accuracy of the messages being purveyed, or their credibility (‘receptor’). Similarly, the 
effectiveness of the use of authoritative tools depends not just on the availability of coercive 
mechanisms and their enforcement but also on target perceptions of government legitimacy or 
the firmness and justice of the use of force or its threat. Similarly, the effective use of treasure 
resources depends not just on the availability of government funding but also on target group 
financial need and especially their receptivity to government funding or their cupidity. Likewise, 
the effective use of organizational tools depends on both the existence of personnel and other 
organizational resource and target group perceptions of government competence in the deploy- 
ment and training of personnel to provide services and rules. 

These are important considerations in the use of tool and especially in their calibration. Thus, 
the use of authority-based tools such as laws and regulations, for example, involves consider- 
ations of legitimacy on the part of targets but must not overreach or overburden the extent of 
legitimacy which a government enjoys (Suchman 1995; Hanberger 2003). If a policy measure 
does so, it most assuredly will require much monitoring and enforcement activity in order to 
be even minimally effective, involving large administrative costs and burdens which may well 
undermine its own efficiency and effectiveness considerations, as has occurred in the past in 
many countries in areas such as marijuana or alcohol prohibition (Issalys 2005). The exact man- 
ner in which a tool is calibrated is affected by a variety of implementation issues, ranging from 
the level of resources available to governments — such as the nature of financial constraints on 
subsidy levels — to their knowledge of the degree to which different levels of resource use affect 
the intensity of mechanism activation. 


Group-Level Mechanisms 


Although often pitched purely at the level of individuals, many of these same mechanisms also 
operate at the more collective or group level (Olson 1965; Buchanan and Tullock 1980; Riker 
1986) (see also Figure 5.4). That is, this same basic logic can be applied to groups or collections of 
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Figure 5.4 Examples of Individual- and Group-Level First-Order Mechanisms 
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individuals who enter into coalitions or act independently in order to pursue collective aims and 
goals, including influencing the behaviour of government towards them. 

Such groups are sometimes viewed as mere aggregates of individual preferences with no 
interests or aims beyond those of their members (Olson 1965), although more careful study has 
shown many more complex motivations and proclivities exist at the collective or organizational 
level which are not reducible in such a fashion (Halpin and Binderkrantz 2011). These mecha- 
nisms include the propensities of groups to search for new issues or retain existing issue orienta- 
tions, whether they prefer to specialize or generalize in issue orientations and the nature of their 
membership appeals (Nownes and Neeley 1996; Halpin et al. 2018). 


Structural- or Subsystem-Level Mechanisms 


A third set of mechanisms is that which concerns the structure of policy subsystems and how 
they change. This set of mechanisms is quite different from the individual- and group-level ones, 
which are the typical subjects of mechanistic analysis. 

This third set of mechanisms is activated by policy tools, especially ‘procedural’ ones which 
affect the manner in which individual and groups act and interact in attempting to affect 
policy outcomes (Howlett 2000, 2011). A sizeable literature in the policy sciences has noted 
the importance to policy outputs and processes of two aspects of subsystem structure: namely, 
the number of types of actor arrayed in a subsystem or network and especially their ability to 
block off or close off entry of new types of actor, as well as the nature of the ideas which cir- 
culate within such subsystems. That is, changes in the ends of policies, be they conceptual or 
practical, require new ideas to be incorporated into policy-making processes (Sabatier 1999; 
Campbell 1998; Blyth 1997; Hall 1993), meaning such ideas have to be able to penetrate into 
the policy communities and networks which control or dominate policy discourses. Similarly, 
another sizeable body of policy research links changes in the conceptual aspects of policy- 
making simply to the ability of actors in policy subsystems to achieve and retain ‘monopoly’ 
or hegemonic status within them (Baumgartner and Jones 1993; Hoberg 1996; Jacobsen 1995; 
Pontusson 1995). 

Like any kind of networks, policy subsystems are composed of nodes and links. Manipulat- 
ing nodes and links — adding, subtracting and changing them — thus constitutes a set of triggers 
which activate a variety of mechanisms at this network level, including the willingness of policy 
actors to enter into relationships with other proximate actors in the network (rather than more 
distant ones) or their desire to act as leaders, entrepreneurs or brokers between other actors and 
governments. 

Hence, there is a third major type of policy behaviour with a specific set of mechanisms, 
which policymakers can and do activate, which are structural ones (see Figure 5.5). 


Second-Order Mechanisms: The Family of Policy Feedback 


All the aforementioned mechanisms are ‘first-order’ ones which directly affect actor or system 
behaviour. But there are also policy mechanisms activated by policy tools which do not directly 
produce effects at the aggregate level but rather involve feedback processes affecting further tool 
choices and policy efforts. These effects require the activation of other types of mechanisms, 
which can be defined as ‘second-order’ mechanisms. 

Policy feedback processes are one such mechanism, for example, and can be either positive 
(reinforcing existing behaviours) or negative (altering them) (Pierson 1993, 2000a, 2000b). Posi- 
tive policy feedback effects often develop lock-in effects over time through increasing return 
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Figure 5.5 Links in the Design Chain — Structural Level 


mechanisms that both freeze the elements of a specific policy and stabilize them on a specific 
path to be followed (North 1990; Pierson 2004; Moynihan and Soss 2014). Thus, they can be 
considered self-reinforcing. Negative feedback avoids the expected outcomes and can advance 
self-undermining policy reform and change processes (Weaver 2010). 


Conclusion 


All in all, many lessons emerge from the study of policy instruments as activators of different 
mechanisms and effects. By focusing on the underlying mechanisms, a mechanistic approach 
can thus help policy design better open the ‘black box’ of policy behaviour and compliance and 
intervene in it. 
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WHAT TOOLS DO 


Policy Targeting and Behaviour 


Michael Howlett 


Introduction: The Utilitarian Roots of Much Thinking 
About Policy Tools 


It is critically important for policymaking that the behaviour resulting from policy activity and 
the expenditure of governing resources matches that anticipated prior to deployment (May 
2004; Kaine et al. 2010; Duesberg et al. 2014). Policy tool use and behavioural expectations are 
thus linked in the sense that the use of policy tools involves implicit or explicit assumptions and 
expectations about the effect the tools will have upon those impacted by them. 

In other words, ‘tools have targets’ (Weaver 2009a, 2009b, 2010). That is, not just ‘targets’ 
in the sense of policy aims and goals and their measures, such as reducing petty crime by 10% 
in a city (Boswell 2014), but ‘targets’ in the sense of individuals and groups whose behaviour 
is expected to be affected by policy activity (Ghosh et al. 2016). Regardless of whether those 
targets are purely social constructions with few empirical referents (Schneider and Ingram 1993, 
2005) or if they reflect more objective assessment of the actual behaviour of relevant groups 
of policy actors, it is critical for effective policymaking that actual target behaviour matches 
expectations. 

‘Compliance’ thus links policy design, tool use and tool choice together and is critical to 
policy success and failure (Weaver 2009b; Lynn 1986; Schneider and Ingram 1990; Shafir 2013). 
However, despite the fact that ‘compliance’ with government intentions has been a significant 
issue in areas such as regulatory studies for many years (Feeley 1970; Etienne 2011; Meier and 
Morgan 1982; Rodgers 1975; Mulford and Etzioni 1978), this aspect of policy design has only 
rarely been systematically examined in the literature on policy tools (Grabosky 1995; Weaver 
2009a, 2009b, 2013, 2015; Winter and May 2001). 

And even when it has been examined, the lessons learned from empirical studies have only 
rarely been related to the principles behind the effective use of particular kinds of policy instru- 
ments (Duesberg 2014; Corner and Randall 2011) and to decisions to use one or more of one 
particular type rather than another in putting together a policy or programme (Tan and Low 
2011; Taylor et al. 2013). Instead, studies of policy tools have often been developed with only 
the most rudimentary and cursory knowledge of how those expected to be affected by the 
policy are, in fact, likely to react to it (Lewis 2007; Corner and Randall 2011; Taylor et al. 2013; 
Duesberg 2014). 


DOI: 10.4324/9781003163954-8 73 


Michael Howlett 


It is often simply assumed, for example, that policy ‘targets’ are rational self-maximizers, cal- 
culating their best interests hedonically in deciding whether or not to comply with the demands 
of government instruments and mechanisms such as regulation, laws and subsidies (Stover and 
Brown 1975; Weaver 2014; Jones et al. 2014; Duesberg et al. 2014; Araral 2014; Maskin 2008). 
Hence, much work in this area is often focused around the idea of “getting incentives right’ or 
calibrating incentives and disincentives, often financial, to achieve expected levels of compliance 
and outcomes rather than on examining other, more normative or culturally determined aspects 
of target behaviour. 

This tendency, however, has changed somewhat in recent years as scholars and practitioners 
alike, many under the influence of behavioural economics, have come to appreciate that mem- 
bers of the public and other policy actors often predictably behave in less than perfectly rational 
ways (Ariely 2010; Thaler et al. 2010; Thaler and Sunstein 2009; Mulgan 2008; Bason 2014). 

This work has led to renewed interest in compliance and in the policy design issues sur- 
rounding it. Weaver (2009b, p. 5), for example, has enumerated some of the various “compliance 
problems’ or ‘barriers’ to compliance which governments face when putting their policies into 
practice. These indeed include incentive and sanction problems where positive and/or negative 
incentives are insufficient to ensure compliance but also monitoring problems where target com- 
pliance may be difficult or costly to monitor, resource problems where targets lack the resources 
to comply even if they want to, autonomy problems where targets do not have the power to make 
decisions that comply with policy even if they want to, information problems where targets lack 
information that would make compliance more likely and attitude and objectives problems where 
targets are hostile/mistrustful towards providers or programs. As Weaver and others have sug- 
gested, these and other similar compliance problems need to be the subject of more investigation 
in order to determine what is needed for effective policy designs that overcome them. 

But if target behaviour is not purely utilitarian, then what is it, and how can it best be antici- 
pated and linked in policy designs to the efficient and effective attainment of government goals? 

This is not a trivial issue in policy theory and practice. Is the best way to encourage an 
increase in birth rates, for example, to provide subsides which might tip the balance of a woman’s 
or family’s calculations of the affordability of children? Or is it more effective to promote family- 
centred events and activities in public service announcements and movie, television and other 
entertainment placements which promote the notion of home life and the pleasures of children 
and family (Lichtenstein and Slovic 2006)? Or both? In what order or sequence? 

Policy tool considerations built around the first orientation can involve debates and discus- 
sions around the appropriate use of particular kinds of financial tools, such as providing more 
widely distributed and available subsidized day care and better local schools rather than around 
how much of a direct subsidy to a parent through the use of tax incentives or cash grants will 
promote higher levels of child birth and larger families (Woodside 1979). But the second may 
involve a much greater range of activities and tools, such as the use of movie theatre and TV 
public service advertisements and educational programmes in schools and elsewhere, rather than 
simply the provision of new services or subsidies. And whether both work in conjunction with 
each other or at cross purposes is unknown. 

Constructing effective compliance regimes is a significant aspect of policy design and requires 
more systematic analysis and understanding of the motivations of policy targets than exist at 
present, ones which would allow better matching of tools and target behaviour right at the 
outset of policymaking, rather than relying on utilitarian conceptions of compliance and non- 
compliance behaviour, which can lead to inappropriate tool choices difficult to alter once in 
place (Weaver 2009a, 2009b; Braithwaite 2003; Schneider and Sidney 2009; Chatterton and 
Wilson 2014; Pierce et al. 2014). 
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Beyond a Compliance-Deterrence Logic in Policy Design: 
Contemporary Challenges to the Utilitarian Foundations 
of Policy Formulation and Design 


As the previous chapters have outlined, the immediate aim of most public policy is to invoke 
behavioural change in some elements of the population in order to have them comply with gov- 
ernment goals. This is done for many reasons, be it promotion of public safety through sanctions 
on criminal behaviour or the reduction of health-care expenses by encouraging healthy living. 
If perfect compliance of targets with governments aims occurred automatically, of course, there 
would be little purpose in undertaking state activity beyond providing the targeted populations 
with information about government goals so that the expected behaviour could then simply 
occur. Since this rarely, if ever, happens, much policy activity, both in terms of policy formula- 
tion and implementation, is about designing programmes and invoking policy tools which can 
encourage behaviour in the direction desired by government and discourage it in others. 

In other words, compliant target behaviour is expected to be achieved through the expendi- 
ture of governing resources in the form of the deployment of specific combinations of substan- 
tive and procedural policy tools which are aimed at modifying or redirecting specific kinds of 
behaviour towards that which is congruent with government goals, aims and ambitions (Ander- 
son 1977; Baldwin 1985). In most circumstances, this means that a substantial part of govern- 
ment policy formulation and design activity is to determine whether or not a target is likely 
to comply with an intended government action and intention and whether such compliance 
will be reluctantly or freely given and by whom (Scholz 1991). Desired changes can be large or 
small, and the expectation of compliance can be rapid or gradual. But in all cases, some changes 
in behaviour in a direction congruent with government aims from some groups of actors is 
expected to result from the utilization of state resources in policy tool deployment. 

Why compliance is not always forthcoming, even when the deployment of government tools 
and resources occurs, is thus a key question in the policy sciences, one which can be addressed 
through efforts to better understand the behavioural underpinnings and conditions of successful 
and failed policies and the study of the kinds of designs and activities which have proven more 
likely to attain success with minimal effort and expenditure (Feeley 1970; Mulford and Etzioni 
1978). 


The Utilitarian Roots of Much Policy Design Thinking: 
Compliance-Deterrence Logics in Policymaking 


Much policy design thinking has been informed by a utilitarian way of thinking about public 
policy compliance, one which has been generally pervasive in the policy sciences from the 
very founding of the discipline (Tribe 1972; Banfield 1977; Shafir et al. 1993). The notions 
of government legitimacy and the ‘compliance-deterrence logic’ associated with this orienta- 
tion towards compliance continue to dominate the discipline, despite the emergence of more 
nuanced thinking about these subjects in recent years, linked to the rise of behavioural econom- 
ics (Oliver 2015). 

In their original ‘compliance-deterrence’ form, legitimate taxes and rules are often simply 
expected to be paid and obeyed by the majority of citizens and targets, with penalties and fines 
established to punish non-compliance in such a way as to ‘deter’ a minority of noncompliers and 
prevent the spread of any unwanted and illegitimate scofflaw and evasive free-rider behaviour 
to the majority (Doern and Phidd 1983). Utility calculations were extended to the calibration 
of penalties and fines, with these set at such a level as to discourage or punish those who might 
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contest the legitimacy of government actions or seek to evade compliance (Lowi 1966; Balch 
1980). 

This approach relied on several traditional utilitarian presuppositions, such as the presence 
of perfect information and reciprocal risk and benefit valuations on the part of policy targets, 
which are difficult to sustain in practice. And even in theory, some aspects of this approach have 
been undermined in recent years by developments associated with the findings of behavioural 
economics and the translation of these insights into the policy sciences (Thaler and Sunstein 
2009; Selinger and Whyte 2012; Liu et al. 2014). This approach has questioned many traditional 
utilitarian presuppositions and led to the articulation of different kinds of policy analysis and 
design principles associated with the design of ‘choice architectures’, such as emphasizing the 
important role played by behaviours such as framing or the manipulation of defaults in affecting 
levels of compliance (Room 2013; John et al. 2009). 

As is well known, this approach has promoted the design and adoption of a different set 
of tools than those linked to traditional compliance-deterrence logics. These tools are linked 
to modest behavioural modification through the provision of ‘nudges’ and other types of cues 
which can engender almost subliminal compliance with government intentions by, for example, 
squeezing highway warning lines together to encourage braking by providing the appearance of 
greater speed than in reality (Thaler et al. 2010; Liu et al. 2014; Lehner et al. 2016). 

While this latter work disputes the idea of the presence of ‘perfect rationality’ among policy 
targets, which often implicitly colours utilitarian analyses, it still accepts uncritically most of 
the hedonic assumptions of such thinking. That is, even in most behaviourally inspired analy- 
ses, ‘subjects’ are still seen to be motivated to promote pleasure and avoid pain and do so in 
an essentially calculating ‘cost-benefit’ fashion when confronted with the choice of rewards 
or penalties associated with whether or not they comply with government measures (Steg 
et al. 2014), although they may not necessarily be able to do so with perfect information or 
knowledge. 

Other policy-relevant studies, however, have long recognized that the compliance of policy 
targets with government intentions is a much more difficult problem than these utilitarian- 
inspired approaches assume, one which requires a different behavioural logic beyond classical 
compliance-deterrence or newer ‘nudge’ models (Meier and Morgan 1982). Empirical studies 
of the degree of compliance of targets with the exercise of coercive authority on the part of 
governments in the form of the creation and imposition of tools and instruments such as laws, 
regulations and taxes, for example, found this behaviour to involve a normative component as 
well as a utilitarian one (May 2004). And some governments, of course, have devoted much 
time and energy to ‘social marketing’ or the use of enhanced appeals to collective identities and 
social mores (such as altruistic or non-altruistic corporate social responsibility [CSR] voluntary 
codes) with the expectation that increased and improved compliance can be achieved at modest 
or less cost by using different instruments from traditional control-deterrence linked ‘command 
and control’ regulation (6 et al. 2010; John 2013a; Dolan et al. 2012; Tallontire 2007; Steurer 
2009; Campbell 2012). 

Thus, government efforts such as raising tax revenues or enforcing regulations were found by 
Hofman et al. (2014) and others to involve a moral and ethical dimension beyond the individual. 
That is, they were found to involve a private-public collective action problem in that in order to 
be effective, actions taken by governments on the part of the general public, such as levying taxes 
to pay for services, require a prior belief on the part of targets that the use of coercion to set and 
collect such taxes is legitimate and that it is in the self-interest of citizens to pay them whether or 
not they directly benefit from the expenditures which resulted from this activity (Hofman et al. 
2014). Similarly, many studies found persuasion and education provided superior compliance 
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results to fines and penalties in areas such as farming practices and construction, among others 
(May and Winter 1999). 

Both these groups of studies showed the compliance situation to be a more complex one 
than a purely utilitarian perspective focused on deterrence would have it. That is, even the most 
basic activities of governance, such as collecting taxes and ensuring laws and rules are obeyed, 
involve not just individual hedonic behaviour but considerations on the part of targets and the 
public of issues such as the legality and normative ‘appropriateness’ of government activity and 
rule enforcement, among other things (March and Olsen 1989). These can include, for example, 
considerations of the legitimacy and illegitimacy of government actors and actions in specific 
fields such as constitutional, religious or privacy-related ones but can also run into and involve 
the desires on the part of individuals and groups to earn praise, avoid shame or avoid guilt and 
social opprobrium for their actions, among other things (Cialdini and Goldstein 2004; Beetham 
1991; Weber 1978; Hofmann et al. 2014). 

Moreover, another group of studies has also shown that different kinds of target groups and 
individuals exist or are perceived to exist in terms of government expectations of the nature of 
their compliant or noncompliant behaviours and can be and are treated differently by govern- 
ments in their policy designs. These different groups of targets have different resources, capabili- 
ties and attitudes which affect whether or not they will comply and how and to what extent 
they will not. 

These attitudes and behaviours can be quite complex and rooted in historical and culturally 
specific views of government intentions and the moral and other aspects of the appropriateness 
of compliant and noncompliant behaviour (Wan et al. 2014, 2015). This variation in target 
structure, motivation and compliance behaviour makes policy design a much more challenging 
activity than a simple hedonic utilitarian perspective would have it. It is an even more complex 
situation than behavioural economics would have it as even behaviourally inspired ‘rules’ of 
semi-rational economic calculations are not enough to capture all the considerations of cultural 
and psychological appropriateness cited in this chapter (Knetsch 2011; Koh 2011). 

Governments, of course, have more tools at their disposal than just authority-based ones 
and thus have a range of options available both to implement policy and to promote compli- 
ance, including those linked to education and persuasion of targets in addition to the exercise 
of coercion (Hawkins and Thomas 1989; Hood 1986). This makes policy design much more 
complex than when viewed as a simple compliance-deterrence problem. That is, targets can be 
influenced to behave ‘appropriately’ and comply with government wishes through many means 
in addition to fines and penalties, such as financial incentives as well as efforts at moral suasion 
and education, which governments may have in higher or lower levels of supply (McLeod et al. 
2015; Hood 1983). And some of these effects are quite subtle. Hence, for example, Kallgren 
et al. (2000) and de Groot and Schuitema (2012) note the manner in which norm compliance 
can be affected by the type of ‘message’ sent urging compliance, including its negative or positive 
nature, as well as other factors linked to the character of the underlying norm the message relies 
on (see also Schultz et al. 2007). 


Reconceptualizing Target Behaviour and Policy Design Imperatives 


Understanding whether a proposed action is likely to trigger behavior linked to ‘affiliation’ or 
‘conformity’ with government wishes or will result in various kinds of non-compliance behav- 
iour from outright disobedience to ‘boomerang’ effects encouraging the action policies are aimed 
at discouraging, or vice versa, is a critical aspect of design work and should be a key criterion for 
policy tool evaluation and deployment (Cialdini and Goldstein 2004; Cialdini et al. 2006). 
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Nevertheless, despite this evidence and these concerns, the utilitarian viewpoint has only 
been seriously challenged in the policy sciences in relatively few instances when it has been 
undeniably apparent that target behaviour is motivated by considerations other than utility. This 
is the case, for example, when the logics of appropriateness clearly dominate those of calculation 
in displays of patriotism or religiously inspired altruistic or resistant activity to, for example, war 
and military service (March and Olson 1989, 2004; Tyler 1990, 2013). Or when the continu- 
ance of clearly self-destructive behaviour such as drug, alcohol or smoking addictions, despite 
prohibitions up to and including death penalties and long prison sentences, prove difficult to 
explain, let alone correct, using a purely utilitarian framework (Vimpani 2005; McGoldrick and 
Boonn 2010). Empirically, however, holding such a position has become increasingly difficult as 
consideration and plans for ‘nudging’ and other aspects, behavioural psychology to policymak- 
ing, have served to undermine the utilitarian paradigm and help bring a new nonutilitarian focus 
to contemporary policy studies (Thaler and Sunstein 2009). 

The same is also true of the recent employment of policy tools such as co-production or 
faith-based public service delivery (Alford 1998; Hula et al. 2007; Kissane 2007; Zehavi 2008) 
and, to a lesser extent, in areas affected by the ‘social marketing’ efforts discussed earlier (Pykett 
et al. 2014). These have also undermined confidence in the ability of utilitarian models to cap- 
ture critical aspects of target behaviour responsible for participation in, and compliance with, 
government schemes and intentions and have led to more efforts to better understand these 
phenomena and build them into policy designs. This is the case, for example, with efforts to use 
information-based tools or moral suasion to try to convince citizens to do their duty and refrain 
from, for instance, littering (Grasmick et al. 1991; John 2013b); to ‘do the right thing’ in giving 
up their seats on public transportation to pregnant women, the disabled, the elderly and oth- 
ers less fortunate than themselves; or, as we have seen, to pay their taxes (Stanbury and Fulton 
1984; Bardach 1989; Torgler 2004; Corner and Randall 2011) or engage in time-consuming 
volunteer work with the elderly or parent-teacher associations (Pestoff et al. 2006; Voorberg 
2014; Alford 1998). 

Such approaches have been especially significant in some jurisdictions in recent years, often 
displacing the deployment of regulation and financial incentives, such as the emphasis on behav- 
ioural modification through social marketing and ‘nudges’ developed by the Blair government 
in the UK after 2008 (Chatterton and Wilson 2014). Although some analyses of co-production 
tools try to link their success and failure to issues such as ‘ability’ or ‘capacity to comply’, which 
reintroduces the idea of utilitarianism, albeit in modified form, as capability deficits are said to 
prevent more straightforward hedonic compliance situations from emerging (Winter and May 
2001; Corner and Randall 2011; Chatterton and Wilson 2014; McLeod et al. 2015), this work 
has not been successful in explaining why people take on voluntary labour or why some do and 
some don’t. Consideration of these tools, which rely on an entirely different set of information- 
or organization-based resources and behavioural motivations for their effectiveness, along with a 
very different way of thinking about policy target compliance than typically found in discussions 
of the use of financial or authority-based ones, is in order in a nonutilitarian policy design world 
(Hood 1986). 


Changing Conceptions of the Nature of the Policy 
Tools-Compliance Relationship 


Notwithstanding these latter concerns, recent activity on the part of many governments experi- 
menting with new tools and techniques such as nudging and co-production (Jordan et al. 2003) 
have not only raised questions about the accuracy of the assumptions of self-interested utility 
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maximization that underlie many existing and proposed policy designs and which have domi- 
nated the academic discourse on these subjects, but have also led to new thinking in this area 
(DiMento 1989; Kahneman 1994; Jones et al. 2011). 

This is not to say that existing work done to date on target motivations and policy implemen- 
tation is not useful or cannot help develop a better understanding of policy instrument design 
and choice. Rather, it is to argue that what is needed in the policy formulation, design and 
implementation processes is a better, more systematic and empirically robust and supportable 
linkage of the expected behaviour of policy targets to the full range of policy tools available. This 
is more than simply a scholarly preoccupation with motivational assumptions, as policy design 
should not be limited by, or focus upon, unproven behavioural presuppositions which might 
artificially restrict consideration of what could prove to be very effective policy instruments in 
dealing with complex problems (Chatterton and Wilson 2014). 

The need to move beyond pure utilitarianism or, rather, to return to suppler, earlier notions 
of target behaviour has become increasingly apparent in recent years as governments around 
the globe have moved to pursue new kinds of policy mixes to address continuing problems 
such entrenched poverty, homelessness, drug addiction, crime and many others which have 
eschewed rapid resolution and undermined utilitarian thinking about tools and targets. These 
efforts have cast doubt on orthodox views of compliance and the motivations of target behav- 
iour. Poor experiences with even alternative tools, however, have also prompted a rethinking of 
the relationship between policy tools and behaviour and the need for better and more evidence- 
informed design (Howlett 2014; Moseley and Stoker 2013). 

This work has focused on better understanding both the ‘supply’ and ‘demand’ sides of policy 
tools and incorporating this knowledge into a better understanding of policy formulation and 
design (Howlett and Mukherjee 2017). 


Better Understanding Policy Tools and Governing Resources: 
The Supply Side of Policy Design 


On the supply side, some congruence between tool characteristics and target behaviour must 
exist in order for there to be any impact at all from the deployment of governing tools. Tax- 
onomies of policy tools generated in earlier eras (Tupper and Doern 1981; Hood 1986; Vedung 
1998; Howlett 2000) helped shed light on this relationship by clarifying how the nature of 
the governing or ‘statecraft’ resources employed by different types of tools in their deployment 
affected behaviour (Hood 1995, 1991; Dunleavy and Hood 1994; Riker 1983, 1986; Dunsire 
1993; Salamon 2002). 

Previous chapters have outlined the overall range of policy tools available to governments, 
including both the ‘substantive’ instrument traditionally examined by economists and a range of 
less economically oriented ‘procedural’ ones (Howlett 2000), which can be used to affect interest 
groups and other actor behaviour and highlight the utility of the model put forward by Hood 
(1986). Following Anderson (1977), Hood grouped tools into a small number of categories 
according to whether they rely on the use of ‘nodality’ (or information), authority, treasure or the 
organizational resources of government for their effectiveness (see Table 6.1). Thus, information- 
based instruments can facilitate the provision of information as well as suppress it and can involve 
the release of misleading as well as accurate information (Mueller 1973; Saward 1992). 

Hood’s idea was that each basic category of tool relied on a particular different kind of gov- 
erning resource and that one of the main reasons one tool would be chosen over another was 
supply oriented: that is, that a government would utilize specific kinds of tools deploying the 
resources it had in ample supply or which could be easily replenished (Hood 1983). 
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Incorporating Compliance: The Demand Side of Policy Design 


This is an important insight. However, as the discussion here has shown, in addition to capacity 
issues, ‘demand-side’ considerations are also very significant in policy design. That is, in general, 
each category of tool involves the use of a specific governing resource, but this resource use is 
expected to trigger or lever a specific characteristic or receptor in targets, inducing a certain 
behavioural response. Thus, the effectiveness of the deployment of such tools is linked not just to 
resource availability — a precondition of their use — but also to the existence of different ‘recep- 
tors’ on the part of policy targets, which make them respond in a predictable way to the use of 
this resource when deployed. 

In the case of information use, for example, tool effectiveness relies on both the availability 
of knowledge and the means to distribute it (‘resources’) and also on the target’s belief in the 
accuracy of the messages being purveyed, or their credibility (‘receptor’). Similarly, the effec- 
tiveness of the use of authoritative tools, as discussed earlier, depends not just on the availability 
of coercive mechanisms and their enforcement but also on target perceptions of government 
legitimacy. Similarly, the effective use of treasure resources depends not just on the availability of 
government funding but also on target group financial need and especially their receptivity to 
government funding or their cupidity. Likewise, the effective use of organizational tools depends 
on the existence of personnel and other organizational resource and also on target group percep- 
tions of government competence and fairness in the deployment and training of personnel to 
provide services and rules. 

Table 6.1 presents a model of the behavioural prerequisites which governing tools rely on 
for the effect. 


Table 6.1 Behavioural Needs for Resource Effectiveness 


Tool Type Statecraft Resource Applied Target Behavioural Prerequisite 


Nodality Information Credibility/Trust — willingness to believe and act on 
information provided by government 

Authority Coercive Power/Force Legitimacy — willingness to be manipulated by government- 
invoked penalties and proscriptions 

Treasure Financial Cupidity — willingness to be manipulated by gain/losses 
imposed by governments 

Organization Organization Competence — willingness to receive goods and services 
from government and enter into partnership arrangements 


Source: Howlett (2011) 


These are important considerations in policy design and especially in the calibration of policy 
tools. Thus, the use of authority-based tools such as laws and regulations, for example, involves 
considerations of legitimacy on the part of targets but must not overreach or overburden the 
extent of legitimacy which a government enjoys (Suchman 1995; Hanberger 2003). If a policy 
measure does so, it most assuredly will require much monitoring and enforcement activity in 
order to be even minimally effective, involving large administrative costs and burdens which may 
well undermine its own efficiency and effectiveness considerations, as has occurred in the past in 
many countries in areas such as marijuana or alcohol prohibition (Issalys 2005). 

The same nonutilitarian behavioural logic extends to the use of taxes and subsidies, although 
the behavioural characteristics of treasure-based policy tools is not the same as for authority- 
based ones. That is, such tools achieve their ends not through a legitimacy-coercion matrix 
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of encouragement and deterrence but through the willingness of subjects to be manipulated, 
more or less voluntarily, by financial incentives and disincentives (Surrey 1970; Woodside 1979). 
These tools will only be as effective if those targets are willing to accept financial awards or 
penalties from governments and alter their behaviour accordingly (Braithwaite 2013). Calibra- 
tions of the settings of such tools are often claimed to be undertaken on a purely utilitarian basis, 
but the extent of cupidity or greed on the part of policy targets varies dramatically by group 
and subject matter as governments discover whenever they attempt, for example, to discourage 
cigarette and tobacco use or obesity by discouraging consumption through raising excise taxes 
on harmful products while reducing them on others (Gullberg and Skodvin 2011; Coffman 
et al. 2016). Such actions may work in some cases and products or among some groups, like 
the elderly in the case of tobacco control, but fail in others, such as young women and younger 
people in general (Studlar 2002). 

This is also true of the use and effectiveness of the deployment of information. As the poor 
experiences of the application of some of the insights of behavioural economics and psychology 
to policymaking in the form of ‘nudges’ or informational cues shows, consideration by targets of 
the credibility of messages sent and received and the willingness of targets to trust their contents 
and promises are critical to what type and extent of behavioural response will ensue (Weiss and 
Tschirhart 1994). This is true, to give only one example, of many different public information 
and marketing campaigns in the areas of obesity and the ingestion of dangerous products which 
utilize this resource (Kersh 2015; Padberg 1992). Again, design in this area involves a subtle 
effort to match resource expenditure and target behaviour. Although some treatments of these 
information tools still base their analysis on behavioural assumptions in which manipulation is 
sometimes expected to follow government cues unthinkingly (‘nudges’), this underestimates the 
impact of trust and credibility of the information sent and received, and ‘libertarian paternalism’ 
(Sunstein 2015) may well undermine these efforts at persuasion by leading to a general distrust 
of government (Jones et al. 2014; Wilkinson 2013; Mols et al. 2015; Galizzi 2014; Momsen and 
Stoerk 2014; Carter 2015). Efforts at ‘social marketing’ discussed earlier also feature the use of 
these informational or ‘nodality’ tools (Hood 1986), ones that appeal directly to sentiments of 
collective solidarity and the moral duty of citizens and groups, which invoke values well beyond 
those related to individual utility calculations (Corner and Randall 2011). 

Efforts at ‘co-production’ and co-design or co-management also often aim at redesigning 
service delivery through various forms of partnerships in which some division of labour emerges 
between state and non-state actors (Pestoff et al. 2006; Voorberg 2014; Alford 1998; Braithwaite 
and Levi 2003). These forms of service delivery are organizational tools which utilize state 
personnel and organizational resources to directly or indirectly deliver goods and services. But, 
again, willingness of targets to partner in these activities, as in the case of more traditional exclu- 
sively government-based ones, requires targets to assess the competence of government agents 
to deliver or plan such services in a timely and appropriate way and of their ability to remain 
independent and autonomous in the face of such support (Howlett et al. 2017). 

These are all policy tool choice challenges which are complicated by the fact that policy tar- 
gets come in all shapes and flavors, from individuals with certain kinds of characteristics to orga- 
nizations of various shapes and sizes, histories, backgrounds and memberships. The preferences 
of such individual single targets are always an issue (Lichtenstein and Slovic 2006; Unsworth and 
Fielding 2014), and in all but the simplest situations, governments are faced with more complex 
environments in which they encounter not just one but multiple actors and target groups. What 
works with one group or section of a group may not work as well or at all with another, and it is 
not unusual for a range of governing resources and tools to have to be deployed in order to deal 
with such complex, ‘target-rich’ environments (Reichardt et al. 2016). 
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The policy mixes or bundles of policy instruments which arise in such cases hence typically 
involve not only a number of tools but also a range of motivations across a range of targets. This 
makes the assessment of the motivational structure of a policy realm more complex and difficult. 
It also suggests that rather than think about compliance in the context of single target-single 
instrument dynamics, policy design should centre on multiple target-multiple instrument ones. 


Conclusion: Better Policy Tool Choices Through Behavioural Study 


Traditionally, much compliance theory in economics and elsewhere has embodied a concept of 
‘compliance-deterrence’ (Kaine et al. 2010) based on the hedonic idea that narrow self-interest 
and calculable utility in enhancing pleasure and avoiding pain are the primary motivators of 
compliance behavior on the part of policy actors (Kaine et al. 2010; Stover and Brown 1975), 
with governments enhancing pain and pleasure in efforts to deter specific kinds of activity and 
encourage others. This thinking has led to many considerations of policy design focusing only 
on the calibrations of policy tools — such as the size of penalties or rewards — rather than on 
the nature of the tools themselves and whether the appropriate tool is being used to match the 
nature of target compliance and cooperation in the design situation. 

This chapter suggests that very different and distinct behavioural patterns and motivations 
underlie the effectiveness of each type of policy tool, most of which are not susceptible to 
utilitarian calculations on the part of either targets or designers, and that matching targets and 
tools typically occurs in a multi-target, multi-instrument setting, which makes design itself a 
much more complex task than often assumed outside the corridors of government where these 
complexities are well known. 

The fundamental design problem for governments, then, is not just determining a given gov- 
ernmental resource endowment and calculating the range of prison sentences or the amount of 
fines and subsidies to levy in some situation, based on a utilitarian compliance-deterrence logic, 
but rather to understand on which basis compliance is likely to occur or not with each target: 
that is, to what extent a government enjoys legitimacy, credibility, competence and cupidity 
among each targeted group. 
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PORTFOLIOS OF POLICY TOOLS 
Types of Policy Mixes 


Michael Howlett and Pablo del Rio 


There are a series of questions about how exactly tools fit together, or should fit together, when 
a policy is enacted. In such cases, the instruments are not isolated from each other, and tools in 
such mixes interact, leading to the potential for negative conflicts and synergies. A second and 
related set of issues involves determining how many tools are required for the efficient attain- 
ment of a goal or goals. And a third set of concerns relates to how any optimum figure can be 
attained in practice. 


Introduction: Tool Mixes and Policy Design Studies 


Policy tools, or the instruments or techniques used by government in order to implement policy 
goals (Howlett 2005), have a special place in considerations and studies of policy design. This is 
because this approach is based on the preparation of plans for tool use with a reasonable chance 
of achieving a specific goal or target (Howlett 2004). Choosing policy tools becomes more 
complex when multiple goals and multiple policies are involved within the same sector and 
government, as is very common in many policy-making situations (Doremus 2003; Jordan et al. 
2012; Howlett et al. 2009). 

These latter kinds of multi-policy, multi-goal and multi-instrument mixes — what Milkman 
et al. (2012) call ‘policy bundles’, Chapman (2003) and Hennicke (2004) a ‘policy mix’ and 
Givoni et al. (2012) ‘policy packages’ — are examples of complex portfolios of tools. These mixes 
typically involve much more than functional logics linking tools to a goal but also deal with 
ideological or even ‘aesthetic’ preferences in tool choices and goal articulation, which involve 
trade-offs and bargaining between actors in choosing one set of tools, goals and policies over 
another (Williams and Balaz 1999). This makes their formulation or design especially problem- 
atic (Peters 2005; Givoni 2013; Givoni et al. 2012). Most often, the focus should move from 
the design of specific instruments to the appropriate design of instrument mixes. This is more 
difficult to do when instruments belong to different territorial/administrative levels. 

Key design questions about such portfolios with which contemporary scholars and practition- 
ers grapple include the issues of avoiding both ‘over’ and ‘under’ design (Haynes and Li 
1993; Maor 2012), how to achieve ‘complementarity’ and avoid ‘redundancy’ or counterproduc- 
tive mixes (Grabosky 1995; Hou and Brewer 2010; Justen et al. 2013a), how to enhance or alter 
mixes over time so that they are able to continue to meet old goals and take on new ones (van 
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der Heijden 2011; Kay 2007) and how to sequence or phase in instruments over time (Taeihagh 
et al. 2013). 

In what follows, several distinctions are drawn between mix types based on the complexity of 
design variables, including the number of instruments, the number of policy goals, the number 
of sectors and the levels of government and sectors involved in the construction and mainte- 
nance of a portfolio. It is argued that if policy design theory is to improve and better inform 
policy practice, then it requires a better understanding of the dimensions of these design spaces 
and the kinds of formulation processes which they encounter (Howlett 2011). 


Problems With Existing Portfolio Analyses 


Although thinking about the design of policy portfolios has been at the forefront of much cur- 
rent research work on policy design (Howlett 2005, 2011; Howlett and Lejano 2013), exist- 
ing studies of such bundles of tools do not use consistent terminology and fail to define the 
dependent variable carefully enough (Howlett et al. 2006). As a result, the cumulative impact 
of empirical studies has not been great, theorization has lagged, and understanding of the mix 
phenomena, despite many observations of their significance, has not improved very much over 
past decades (Chapman 2003; Ring and Schroter-Schlaack 2011). 

Most older literature on policy tools focused on single instrument choices and designs (Tupper 
and Doern 1981; Salamon 1989; Trebilcock and Prichard 1983), and these studies provide only 
limited insights into the complex arrangements of multiple policy instruments which are com- 
monly found in all policy fields (Jordan et al. 2011, 2012; Givoni 2013). Many significant issues 
related to the manner in which tool choices in bundles are made and how tool bundles evolve 
over time affect the propensity for designs to avoid the twin shoals of over- and underreacting 
to problems (Maor 2012; Howlett and Rayner 2007) while incorporating better knowledge of 
both synergistic and counterproductive tool relationships and interactions (Del Rio 2010; Lep- 
lay and Thoyer 2011; Grabosky 1995; Justen et al. 2013b). 

First, there are a series of questions about how exactly tools fit together, or should fit together, 
into a mix. In such mixes, the instruments are not isolated from each other, and tools in such 
mixes interact, leading to the potential for negative conflicts (‘one plus one is less than two’) 
and synergies (‘one plus one is more than two’) (Lecuyer and Bibas 2012; Philibert 2011). In 
such cases, different design principles are required to help inform portfolio structure. Here the 
question of tool complementarity looms large. As Tinbergen (1952) noted, additional tools — 
‘supplementary’ or ‘complementary’ ones — are often required to control side effects or other- 
wise bolster the use of a ‘primary’ tool. Bundling or mixing policy tools together in complex 
arrangements, however, raises many difficult questions for students and practitioners when there 
are significant interactive effects among policy tools (Boonekamp 2006; Yi and Feiock 2012), 
some of which may be very difficult to anticipate or quantify using standard analytical tools 
(Justen et al. 2013a, 2013b). 

A second and related set of issues involves determining how many tools are required for the 
efficient attainment of a goal or goals. This concern has animated policy design studies from 
their outset, and an example of an oft-cited rule in this area originating in the very early years 
of policy design studies is that the optimal ratio of the number of tools to targets or goals in any 
portfolio is 1:1 (Knudson 2009). 

This is a rule-of-thumb design principle towards which Tinbergen (1952) provided some 
logical justification in his discussion of the information and administrative costs associated with 
the use of redundant tools in the area of economic policy. Most observers, however, dispute 
that such a simple situation was ever ‘normal’ and instead argue that combinations of tools are 
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typically found in efforts to address multiple policy goals (Jordan et al. 2012). The issue of 
potentially under- or overdesigning a mix arises in all such circumstances and is made more 
complex because in some instances, for example, arrangements may be unnecessarily duplica- 
tive while in others some redundancy may be advantageous in ensuring that goals will be met 
(Braathen and Croci 2005; Braathen 2007). 

A third set of concerns relates to how any optimum figure can be attained in practice. This 
concern is less a spatial than a temporal one as the existing evidence shows that suboptimal situ- 
ations are very common in many existing mixes which have developed haphazardly through 
processes of policy layering (Thelen 2004; van der Heijden 2011). This is a process in which 
new tools and objectives have been piled on top of older ones, creating a palimpsest-like mixture 
of quite possibly inconsistent and somewhat incoherent policy elements (Howlett and Rayner 
2007; Carter 2012). These processes and change dynamics focus attention on the sequencing of 
instrument choices (Taeihagh et al. 2009, 2013b) and especially on the fact that many existing 
mixes have developed without any sense of an overall conscious design. These kinds of ‘unin- 
tentional’ mixes can be contrasted with ‘smarter’ designs which involve creating new sets of tools 
specifically intended to overcome or avoid the problems associated with layering but which may 
be harder to put into practice (Gunningham et al. 1998; Kiss et al. 2012). 

In other words, intelligent design of policy mixes begins with ensuring a good fit not only 
between packages of tools and government goals and their institutional and behavioural contexts 
at a specific moment in time (Considine 2012; Lejano and Shankar 2013) but also across time 
periods as new instruments appear and old ones evolve or are eliminated. That is, design analyses 
must extend beyond questions of tool synergies and optimal design to consideration of how and 
why mixes change over time and how the processes of policy formulation followed in adopting 
such complex designs take place (Larsen et al. 2006; Kay 2007; Feindt and Flynn 2009). 


Better Defining the Design Space: Vertical and Horizontal Levels 
of Complexity in Policy Portfolios 


Most work on the subject of policy portfolio design fails to define the design space carefully 
enough to be able to distinguish the impact on different design choices of the spatial and tempo- 
ral factors influencing the portfolio design process. Most studies, for example, fail to differenti- 
ate between simple and complex contexts and simple and complex designs and mixes (Howlett 
2004; Howlett et al. 2006). But, as the earlier discussion of the Tinbergen rule illustrated, 
incorporating the level of complexity of a mix is an important characteristic of the problem 
context which principles of portfolio design must take into account. Providing a better model of 
policy design spaces helps reveal some important variations in terms of who makes or is capable 
of making design decisions, as well as on the likely content of that decision in specific contexts 
(Howlett 2011). 

In addressing the issue of design spaces and their impact on policy designs and designing a 
first-order distinction must be drawn between single ‘level’ mixes and those with a more com- 
plex structure. That is, in addition to the ‘horizontal’ issue addressed by many students of policy 
mixes — pertaining to the kind of relationships existing between tools, goals and policies within 
a single level of government and sector of policymaking — a second, ‘vertical’ dimension is pres- 
ent and often ignored in these studies. This vertical dimension involves not just the number of 
instruments, goals and policies found in a mix, but also the number of policy sectors they involve 
and the number of governments active in policy formulation in this area (del Rio 2009). 

Such a framework allows room for many more complex interactions between bundle ele- 
ments than typically envisioned or analyzed in existing studies. That is, conflicts and synergies 
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between tools, goals and policies can be identified both at the horizontal level — for example, 
between different types of instruments and goals within each level of analysis — and/or at the 
vertical: that is, across and between different policy sectors and/or administrative levels. These 
variations have significant implications for both the number and type of actors involved in policy 
design and the processes through which formulation unfolds, as well as for the complexity of 
design itself. While some aspects of horizontal interactions can be addressed in largely technical 
ways — so that, for example, some conflicts can be mitigated just by selecting certain instruments 
over others — in more complex cases, such analyses must be supplemented by other political, 
administrative and organizational logics, and policy formulation processes become more diffi- 
cult. These challenges are multiplied as mixes evolve over time. 

That is, vertical design contexts cutting across sectors and governments require efforts aimed 
at achieving administrative coordination and policy integration suitable to the complexity of 
context, which horizontal mixes generally do not. In the former situation, relevant coordina- 
tion, for example, needs to be in place between different administrative levels and across policy 
subsystems which are not needed in simpler horizontal contexts. The configuration of elements 
in a vertical mix must relate to preferences for different instruments favored in multiple sectors 
and governments rather than just among a single set of actors (Freeman 1985; Howlett 2009). 
And shifts in these preferences over time require changes to existing mixes, which may be more 
or less easy to achieve and require special handling or developmental techniques and analysis. 


Developing a Basic Taxonomy of Policy Mixes 


Developing a typology of policy mixes based on the level of complexity of design spaces is a 
useful first step in advancing design studies beyond their current weak status. Mixes can be 
assessed at a general level by identifying spaces of conflicts, complementarities and synergies 
between policy fields, but those interactions also depend on the type of tools being adopted and 
the specific design elements of the instruments adopted within those policy fields. The choice 
of specific instruments and design elements within interacting policy fields may contribute to 
mitigate conflicts and promote complementarities and synergies or not. Coordination is easier 
under certain instruments and design elements than under others. 

The first key dimension in constructing such a taxonomy relates to distinguishing between 
mixes according to the number of instruments, goals and policies found within the horizontal 
level. Additional scenarios then exist for vertical mixes in situations in which multiple instru- 
ments and goals exist across sectors and governments. As at the horizontal level, at these levels 
tools and goals may complement each other while in others or in some aspects, they might not 
(Hull 2008; Flanagan et al. 2011). 

While relatively simple mix design processes may be dominated by expert actors (Dunlop 
2009) and decided on according to technical or functional criteria (Braathen 2007), moving 
towards multiple goals brings in additional actors such as those arrayed in ‘epistemic communi- 
ties’ (Marier 2008) and involves more sophisticated evidence and ideas than are found in more 
simple contexts (Sanderson 2002). In such multi-level government and governance contexts 
(Hooghe and Marks 2003), different levels of government are likely to have some common but 
also different goals and instrument preferences (Enderlein et al. 2011), and reconciling them 
typically involves the use of the overt political calculus of intra- or intergovernmental bargaining 
and decision-making (Bolleyer and Borzel 2010; Kaiser et al. 2012). 

Increasing complexity from horizontality to verticality brings in cross-sectoral or cross- 
national epistemic actors (Haas 1992), including political ones, and often involves the assessment 
and use of politically contested evidence and criteria (Gilabert and Lawford-Smith 2012). The 
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most sophisticated design spaces involve the most complex design processes and the full range 
of subsystem actors operating across multiple governance levels (McCool 1998; Hooghe and 
Marks 2003). Here, in a context of vested interests, lobbying pressures and intergovernmental 
jurisdictional disputes — fully blown political criteria such as blame avoidance, credit claiming, 
bargaining and log-rolling relevant information (Hood 2010) — are features of policy formula- 
tion, and designs take on new forms and patterns. 

Taking these five aspects of horizontality and verticality into account, and assuming simple 
binary measures of complexity at each level, yields 32 possible configurations of portfolios. This 
complexity can be greatly reduced, however, by restricting analysis to only complex tool mixes: 
that is, eliminating from further analysis half the circumstances whereby only a single instru- 
ment is utilized. Combining both cross-sectoral and multi-governmental vertical elements into 
a single multi-level variable then reduces this to eight basic types (see Table 7.1). 

In this model, mixes can be seen to range from the simplest type, when multiple tools are an 
issue (Type I), to the most complex multi-level, multi-policy, multi-goal type (Type VIII). Four 
of these eight types are ‘instrument mixes’ which involve single policy contexts (Types I, I, V 
and VI), and therefore are less complex than their multi-policy counterparts (Types III, VII and 
VII), which can be termed ‘policy mixes’. 

Are all eight types equally likely to occur? Although much of the literature seems to sug- 
gest that Type I situations are the norm, empirical studies suggest this is not the case (Howlett 
et al. 2006; Hosseus and Pal 1997) and that more complex design spaces and, hence, policy 
portfolios are commonplace and growing. Factors such as the administrative and legislative 
arrangements present in federal and nonfederal systems affect the likelihood of the appearance 
of multi-governmental mixes (Howlett 1999; Bolleyer and Borzel 2010), while increasing 
efforts to promote collaborative or horizontal governance arrangements, for example, will 
affect the number of multi-sectoral and multi-policy situations which exist (Peters 1998; Kop- 
penjan et al. 2009). 


Design Implications Flowing From This Taxonomy 


This model of mix types and design spaces helps overcome the three sets of issues raised here: 
how to avoid both ‘over’ and ‘under’ design, how to achieve ‘complementarity’ and avoid 
‘redundancy’ or counterproductive mixes and how to enhance or alter mixes over time so that 
they are able to continue to meet old goals and take on new ones. Each of these issues is dis- 
cussed in turn next. 


Table 7.1 Basic Typology of Portfolio Designs 


Types 

Dimension I I II IV V VI VII VII 
Multi-level No No No No Yes Yes Yes Yes 
Multi-policy No No Yes Yes No No Yes Yes 
Multi-goal No Yes No Yes No Yes No Yes 


Simple Complex Simple Complex Simple Complex Simple Complex 
single-level single-level single-level single-level multi-level multi-level multi-level multi-level 
instrument instrument policy mix policy mix instrument instrument policy mix policy mix 
mix mix mix mix 
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Avoiding Over- and Under-designing 


Prima facie, the taxonomy set out in Table 7.1 shows that simple Tinbergen-type single-instru- 
ment, single-goal, single-policy, single-government instrument mixes represent only one of many 
possible types of instrument mixes. And this means that the standard Tinbergen design maxim of 
‘one goal-one tool’, proposed as a suitable design maxim to address the issue of instrument opti- 
mality, is unlikely to be put into practice very often, and other principles need to be developed 
to take its place within more complex designs if over- and under-designing are to be avoided. 
These, to a certain extent, now have come to relate to the need to promote synergies and avoid 
counterproductive tools uses which were previously divorced from the optimality issue by the 
assumption of simpler design spaces and rules. 


Avoiding Conflicts and Promoting Complementarities 
and Synergies in Tool Uses 


When multiple tools are involved in a mix, the tools involved and invoked in a mix may be 
inherently contradictory (Tinbergen 1952; Grabosky 1995; Gunningham et al. 1998) in the 
sense that they evoke contradictory responses from policy targets (Schneider and Ingram 1990a, 
1990b, 1993, 1994, 1997, 2005), while other combinations may be more virtuous in providing 
a reinforcing or supplementing arrangement (Hou and Brewer 2010). Although a consensus 
does not exist on the terms and definitions of conflicts, complementarities and synergies (Oiko- 
nomou and Jepma 2007; Oikonomou et al. 2010, 2011), nevertheless, it can be argued that the 
types of interaction found between tools will vary such that in some cases there will be: 


a strong conflict: where the addition of an instrument (X) leads to a reduction of the effect 
of a second instrument (Y) in the combination: 0 < X+Y < 1; 

a weak conflict (partial complementarity) where the addition of an instrument to another 
leads to a positive effect on the combination, but lower than the one that would take 
place if both were used separately: 1 < X+Y < 2; 

a situation of full complementarity where X adds fully to the effect of Y in the combina- 
tion: X+Y = 2; and 

a situation of synergy where adding X to Y magnifies the impact of the combination: 
X+Y > 2. 

(del Rio 2014) 


Effective design would involve avoiding strong conflicts, minimizing weak ones and promoting 
complementarity and synergies. 

Here the idea would be to attempt to avoid conflicts of both types while promoting tool 
combinations which are complementary or synergistic. While this becomes more difficult to do 
as the level of complexity of the design space increases, it remains a central goal of a portfolio 
design. It may be impossible to satisfy all assessment criteria with different instruments when 
more than one goal, policy or government is involved. The best way to address trade-offs and 
conflicts between criteria is to adopt a multi-criteria framework which makes those conflicts 
explicit. This allows policymakers to give weight to those criteria and decide on the trade-off 
according to their preferences. 

Linking tools and goals is a second area in which synergies and complementarities can be 
sought. Criteria such as ‘consistency’, ‘coherence’, ‘congruence’ and level of ‘integration’ have 
been suggested as useful in this area of portfolio design (Howlett and Rayner 2007; Lanzalaco 
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2011; Mandell 2008; Howlett and Rayner 2007; Kern and Howlett 2009). Work on mixes in 
sectors such as climate change mitigation and renewable energy support (del Rio Gonzalez 
2007, del Rio et al. 2011; del Rio 2009, 2010; Boonekamp 2006) lead the way in this regard. 
As del Rio (2009) has argued, design principles to promote integration in complex mixes 
require a broader view of the elements found in policy mixes than is typically found in the litera- 
ture on the subject. That is, appropriate policy evaluation, appraisal and design cannot be con- 
ducted in a narrow context. The focus should not be on the functioning of specific instruments 
with respect to one specific criterion, but rather on the functioning of the whole policy mix 
and the conflicts and synergies with respect to several goals and criteria in this portfolio. This 
is a particular challenge with overlapping policies and governments. What might be regarded 
as conflictive in the interactions within an instrument mix might not be so problematic when a 
broader picture of a policy or governmental mix is considered. But both horizontal and verti- 
cal coordination are very difficult to achieve. There is certainly a role for coordination between 
goals and instruments to mitigate conflicts and to promote complementarities and synergies in 
policy mixes. But the existence of different goals at different administrative levels complicates 
vertical coordination. Different benefits and costs for different constituencies stemming from 
supranational policies may lead to low levels of social acceptability and considerations of political 
feasibility. Different goals may create winners and losers at lower administrative levels and thus 
lead to unacceptable distributional effects. All these factors must enter into design considerations. 


Promoting Patching as Well as Packing in Portfolio Design 


Finally, there is the issue of temporality and how to handle it. Considerations on how to over- 
come temporal legacies in existing portfolios range from thinking about designing in a situation 
which Thelen (2003) describes as ‘replacement’ — that is, one in which design occurs de novo, 
and all previous regime elements have been swept away or do not exist — and in situations char- 
acterized by ‘layering’ — in which design occurs within the context of difficult-to-remove policy 
elements (Rayner et al. 2013). 

That is, at least two distinct design techniques emerge here as formulation efforts may take 
the form of policy ‘packaging’ — that is, the creation of new mixes — or ‘patching’, in which only 
selected aspects of existing mixes are altered. Recognizing the drawbacks of layering, conver- 
sion and drift as often promoting unintentional mixes, many critics have increasingly argued for 
the promotion of complex policy mixes through replacement. However, multiple policy tool 
portfolios which have evolved over a long period time through processes of incremental layering 
cannot easily be replaced. Policy ‘patching’ is a more realistic design modality in such contexts 
and, if done properly, with a clear eye on promoting coherence and integration in complex 
environments, can achieve complex and ambitious policy goals in as efficient and effective a way 
as those designs which are consciously created as interlocking packages of measures (Feindt and 
Flynn 2009; Kay 2007; Howlett et al. 2015). 


Conclusion 


The aim of this chapter has been to develop the main elements of a theoretical and method- 
ological taxonomy which can help clarify the different types of policy portfolios which are cur- 
rently often ignored or improperly juxtaposed in the literature on the subject. This was done 
in an effort to provide the basis not only for better designs but also for improved considerations 
of the formulation processes and actors involved in such complex policy-making efforts. The 
discussion thus contributes to efforts currently being made to assess the success or optimality of 
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complex policy mixes (Mandell 2008; Howlett and Rayner 2013; del Rio 2014) and advances 
the project of revitalizing policy design studies urged by Howlett and Lejano (2013). 

The multidimensional nature of policy mixes is a phenomenon which has been ignored in 
most of the policy instrument choice and policy design literature, resulting in a lack of clarity 
and difficulties associating different kinds of actors and evaluation criteria with mixes (Leutz 
1999; Justen et al. 2013a, 2013b) and the continual use of outdated or inappropriate design 
maxims in their construction, which significantly enhances the potential for over- and under- 
designing. Even with only three main portfolio dimensions — goals, policies and levels — the 
design situation is more complex and nuanced than is normally depicted in the existing policy 
design literature. 

The chapter argues that complex policy mixes inherently involve interactions between the 
different instruments of which they are composed, in the form of either conflicts or syner- 
gies. These can be defined as horizontal — between different types of instruments, policies or 
governments — or vertical — between different levels of goals, policies and government. These 
two dimensions each contain a number of elements and a large number of possible permuta- 
tions. However, it is possible to refine significant mix types and design spaces to eight basic types: 
four relatively simple instrument mixes and four more complex policy mixes. 

Mitigating the conflicts and encouraging synergies within these mixes through effective pol- 
icy design first requires recognizing these different design spaces and their implications for what 
is being designed and by whom (Howlett 2013). Only then can efforts take place to enhance 
relevant horizontal and vertical coordination between and within different administrative lev- 
els and sectors relating to different instruments, goals and policies contained within a mix. 
The typology of outcomes set out in Table 7.1 suggests an increasingly complex environment 
for policy formulation as the complexity of portfolio parameters increases, ranging from rela- 
tively simple single-instrument mixes to the multi-level, multi-goal and multi-policy bundles of 
higher-numbered types (Keast et al. 2007). 

The potential for complementarity and coherency effects to be actualized increases in level 
of difficulty as more goals, policies and governments are involved in a ‘bundle’ or ‘portfolio’, 
and the number of actors and types of evidence used in designing correspondingly increases in 
complexity and variability (Escribano 2013). Developing such a multidimensional typology of 
policy portfolios, however, is a needed step, helping clarify several outstanding issues in portfolio 
design which eluded existing literature on the subject. The typology allows us, for example, to 
begin to generate a multi-level model of tool selection and design showing how the problems 
(conflicts) in horizontal interactions can be mitigated by design principles such as coordinat- 
ing targets, instruments and/or design elements and suggests that, in many instances, designing 
through patching will produce superior results to design through packaging or replacement. 
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UNDERSTANDING PRIMARY AND 
SECONDARY RELATIONSHIPS 
AMONG POLICY TOOLS 
IN POLICY MIXES 


Michael Howlett, M. Ramesh and Azad Singh Bali 


A recent resurgence of interest in policy design has fostered renewed efforts to better understand 
how specific combinations of policy tools arise and shape policy outcomes. However, to date, 
these efforts have been stymied by under-theorization of the different purposes to which tools 
are directed in policy mixes and a corresponding failure to acknowledge the different purposes 
towards which different tools in a mix are directed. The central argument of this chapter is that 
existing frameworks do not adequately recognize the complexity of contemporary policy tool 
mixes, especially their hybrid and multilayered features, and how procedural and substantial tools 
operate and interact together in such mixes in priority and supportive roles. To close these gaps, 
we propose a revised tool framework that distinguishes between first- and second-order aspects 
of instruments used in policy mixes and highlights the particular salience of procedural tools 
within them. The framework, we argue, is simple to operationalize and thus aids understanding 
of the complex architecture of mixes, allowing closer examination of the conditions and factors 
which shape policy performance. 


Introduction: The Policy Tools Approach 


The most commonly adopted theoretical frameworks in contemporary policy sciences — such as 
the advocacy coalition framework, punctuated equilibrium theory, the multiple stream approach, 
narrative policy framework — all focus on the input side of the policy process, especially the roles 
played by different kinds of actors — policy entrepreneurs and brokers, discourse coalitions, 
epistemic communities and others — in policy processes (Howlett et al., 2020). This is useful in 
understanding policymaking but less so when it comes to analyzing policy content and effects. 
There is a parallel stream of thinking in policy studies, however, that focuses on the outputs of 
policymaking, analyzing in detail the content of adopted decisions, which is much more useful 
in this pursuit (Hood, 2007). This is the “tools approach” to policy studies, in which the outputs 
of the policy process are conceptualized as comprised of specific policy “tools” or “instruments” 
deployed in order to attain specific policy goals and objectives. In this latter approach, scholars 
work backwards from outputs to inputs in assessing policymaking less as an open-ended or 
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boundaryless struggle between ideas or interests than as a process of choosing or selecting spe- 
cific tools expected to achieve policy goals (Peters, 2018; Howlett, 2019). 

Understanding the use of “bundles” or “portfolios” of instruments rather than single tools is a 
focus of much current work in instrument studies and helps elucidate, promote and design more 
effective governance strategies (Doremus, 2003; Sterner, 2003). These kinds of multi-policy, 
multi-goal and multi-tools mixes — what Milkman et al. (2012) call “policy bundles”, Chap- 
man (2003) and Hennicke (2004) “policy mix” and Givoni et al. (2013) “policy packages” — are 
examples of complex portfolios of tools whose study has been, to date, only very partial and 
exploratory (Rogge and Reichardt, 2016). 

The resurgence of interest in policy design has fostered efforts to better understand how such 
combinations of policy tools arise and shape policy outcomes. However, these efforts have been 
stymied by under-theorization of the different purposes to which tools are directed in policy 
mixes. One key issue with such mixes, for example, is the relationship(s) existing among the dif- 
ferent tools in a mix or package. Thus, for example, taxes are typically layered with nodal tools 
such as nudges that help effect compliance. Similarly, public health financing programs which 
cover citizens are often combined with tax incentives to encourage individuals to purchase 
supplementary private insurance. Significantly, in both these cases, some tools are “primary” — in 
this case taxation and public financing — while others are “secondary” — nudges and tax incen- 
tives. Health insurance co-payments such as user fees and deductibles to offset costs or reduce 
malfeasance associated with health insurance are another example of secondary policy tools 
commonly deployed along with public financing in health-care policy mixes (Blomqvist, 2011). 

In general, current studies have focused on better identifying and measuring only the overall 
composition of policy portfolios or mixes (e.g., Kern and Howlett, 2009; Schaffrin et al., 2014, 
2015) in specific policy sectors and have not addressed in detail the key features of the inter- 
and intra-tool relationships they contain (McDermott et al., 2009, 2010; Auld et al., 2010; 
Schmidt and Sewerin, 2019). For an exception, see Hou and Brewer, 2010). The key question 
for most existing studies has been simply to describe what kind of policies or policy mixes exist 
and begin to grapple with the question of how overall policy configurations and specific policy 
components have developed and changed over time (e.g., Howlett and Lejano, 2013; Howlett 
and Rayner, 2013). 

The central argument of this chapter is that existing frameworks do not adequately recognize 
the complexity of contemporary policy tools in policy mixes, especially their hybrid and multi- 
layered features, and how both procedural and substantial tools operate and interact in priority 
and supportive roles. To close these gaps, we propose a revised tool framework that distinguishes 
between first- and second-order aspects of instruments used in policy mixes and highlights the 
particular salience of procedural tools within them. The framework, we argue, is simple to 
operationalize and thus aids understanding of the complex architecture of mixes, allowing closer 
examination of the conditions and factors which shape both policy dynamics and performance. 


Understanding Policy Mixes 


Early studies of policy tools focused on describing the types of tools governments use to advance 
their goals and the criteria which affect the choice of individual tools (Lowi, 1972; Hood, 1983; 
Linder and Peters, 1989; Peters and van Nispen, 1998; Howlett, 2000; Salamon, 2002). A more 
recent wave of studies on policy design, however, has brought about a renewed interest not only 
in such questions but also in new issues such as how tool choices evolve over time and, espe- 
cially, how individual tools operate when combined with others, sometimes in very complex 
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multi-level, multi-policy and multi-tool “policy mixes” or “portfolios” (Howlett and del Rio; 
Howlett and Lejano, 2013; Howlett et al., 2015). 

It has increasingly been recognized that most contemporary policy problems are too complex 
to be addressed by a single tool but rather require the use of multiple tools based on different 
governing resources (Hood, 2007). As Peters (2018) recently reminds us, there are now very few 
policies composed of discrete and standalone policy tools. Mixes or “combinations of different 
kinds of policy tools (market-based, hierarchical, network and others) whose exact configura- 
tion changes from location to location” (Rayner et al., 2017, p. 473) are much more common. 

This is an important observation and insight. Thus, social insurance, for example, a com- 
monly used tool to finance health care around the world has commonly been understood 
as a purely financial tool. But, in practice, social insurance requires an administering agency 
(organizational tool), legislation mandating participation (an authority tool) and the availability 
of public information on the scheme in addition to these treasure-related activities. Another 
example can be found in how to manage traffic congestion. Many countries now “bundle” 
licenses, congestion pricing and user fees (Wu and Ramesh, 2014) so that traffic management 
policy is, at minimum, a three-instrument mix. And congestion pricing, commonly under- 
stood as a fiscal tool, also requires legislation (an authority tool) and an administering tax 
agency (an organizational tool) and has to be based on accurate and easily transmissible traffic 
data (an information tool) in order to operate effectively, adding to the complexity of such 
portfolios. 

These mixes can develop in a conscious, intentional way or through a more haphazard pro- 
cess, and studies of mixes have begun to examine how policymakers choose particular instru- 
ments and whether and how they change previous choices (Capano and Lippi, 2017) and how 
policy actors aggregate around specific policy instruments to form “instrument constituencies” 
promoting certain tools, often regardless of the nature of the problem (Voß and Simons, 2014; 
Béland and Howlett, 2016), Studies have also looked at the political and policy effects achieved 
by adopting specific policy instruments: for example, creating coalition effects for actors benefit- 
ing (or not) from policy efforts (Borras and Edquist, 2013; Bressers and Klok, 1988; May et al., 
2005). 

These studies, among other things, have identified several aspects of mixes and their com- 
positive elements which constitute important attributes of these kinds of tool portfolios, affect- 
ing their ability to attain government goals and ambitions (see Table 8.1). 

Yet despite this work, there remain several important gaps in our understanding of the nature 
of policy composition and dynamics between and among tools within a policy mix which 
negatively affect the ability of scholars to generalize about them and practitioners to design and 
deploy them (see Capano and Howlett, 2020). 

Recent efforts to measure policy dynamics, for example, while useful, often mischaracterize 
mixes and do not offer a compelling understanding of how choices are arrived at within the 
spectrum of tools involved or how such choices create an operational bundle of policy tools. 
For instance, many studies of mixes have stressed the importance of policy density (Knill et al., 
2012), a measure which focuses on the number of tools in a policy mix but ignores the differ- 
ent emphasis or weight placed on specific tools within a policy mix. Moreover, it ignores the 
institutional architecture that supports each tool and conditions its performance. Similarly, other 
studies have attempted to combine the measure of density with one related to policy intensity, 
which often stresses the coercive strength of individual tools. This does not, however, take into 
account the interactive or synergistic effects different tools have when they are bundled together. 

Moreover, empirical studies continue to show how governments typically rely on multiple 
and offsetting kinds of policy tools to achieve a stated goal (Wu and Ramesh, 2014) mixing, 
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Table 8.1 Attributes of Policy Mixes 


Attribute Description 

Coherence The extent to which policy goals and instruments are consistent over time 
(Howlett & Rayner, 2007) 

Complementarity The extent to which there are synergies or “positive interactions” between 


various policy tools and components within an existing mix (Howlett, 2018) 
Completeness The deliberate inclusion of procedural elements in a policy mix with the goal of 
managing governance failures in service delivery (Saguin, 2020) 

Goodness of Fit The extent to which individual instruments and their settings are compatible with 
governance style and the broader political context (Howlett et al., 2015) 
Degrees of Freedom The scope for future changes/calibrations to the instrument (Howlett et al., 2015) 


Volatility The likelihood or propensity of mixes to contain instruments that that are 
inherently unstable (Howlett, 2019) 
Intensity The specific content of the instruments (i.e., the level and scope of government 


intervention) measured through six indicators (objectives, scope, integration, 
budget, implementation and monitoring) (Knill et al., 2012; Schaffrin et al., 


2015) 

Density The number of instruments in a policy mix or used to manage a policy area 
(Knill et al., 2012) 

Explicitness The extent to which instruments in a mix define a desired direction of 


behavioural change relative to a given technical policy goal (Thomann, 2018) 


Source: Adapted from Bali, Howlett and Ramesh (2021) 


or blending many elements such as the procedural or substantive nature and impacts of tools 
(Howlett, 2000). In health care, for example, procedural tools such as budget ceilings sharpen 
commonly used fiscal tools such as capitation payments, and, in turn, fiscal tools (e.g., subsidies, 
fees, penalties) are used to sharpen the impact of regulatory interventions (Liu, 2003). Certain 
tools also allow the government leverage to steer actors in a sector and are must be reconciled 
with tools which direct specific kinds of actors or actor behaviour, such as regulation which 
might prohibit or incentivize certain kinds of activity. For example, public ownership of hos- 
pitals can allow governments to set standards on waiting times, costs and clinical outcomes at 
public hospitals. The absence of such ownership consequently may deny the government this 
leverage of the sector and actor behaviour (Ramesh and Bali, 2019, 2021). 

Empirical research on mixes has not yet sufficiently developed with regard to measuring these 
interactive effects of policy tools, and there is still disagreement about the definition and mean- 
ing of concepts such as mix consistency and coherence (Rogge and Reichardt, 2016), which 
this chapter sets out to resolve. 


Policy (De)Composition as a Continuing Problematic Element 
in Policy Design Studies 


Surprising as it may sound, the discipline of public policy has for decades largely evaded the 
question of what is a “policy”, often conflating it with a “decision” to implement an action 
without specifying its exact nature and content (Howlett and Cashore, 2009). A “decisional” 
conception of policy is problematic, however, not the least for being unable to distinguish ade- 
quately between small and large decisions, such as between purchasing stationery and reforming 
the education system, but also because a decision, in itself, often amounts to approving some 
set of policy measures without actually describing or capturing the nature of those measures. 
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A major development on this front was Hall’s (1993) initial disaggregation of the elements 
which made up a policy, distinguishing between three basic components — goals, tools and 
calibrations — into which policies could be decomposed. This process was enriched by Cashore 
and Howlett’s (2007) modification of Hall’s model to include six constituent components of 
public policies: goals, objectives, specifications with respect to policy ends, instrument logic, 
tools and calibrations for policy means, as depicted in Table 8.2. 

While it is apparent that policies are composed of all six elements, researchers typically con- 
tinue to describe only one or a few of the macro- or micro-level policy elements and overlook 
the interrelationships between these elements: that is, how changes to one component affect the 
overall character and effects of the policy. 

For example, comparative studies of policy regimes typically focus on institutional arrange- 
ments used to organize policy subsystems. These studies are typically pitched at the macro level 
and emphasize how ideas such as attitudes towards societal risk pooling and ideas regarding 
the role of the state in the health sector, state and nonstate actors in the provision and financ- 
ing of health care, and institutions (e.g., social security organizations, social contracts and the 
like) shape policy choices (Esping-Andersen, 1996; Moran, 2000; Harris and Milkis, 1989; 
Jochim and May, 2010; Preker and Harding, 2003). This has contributed to policy debates 
on national “models” of policy focusing on broad generalizations around existing macro-level 
arrangements while downplaying tools choices and configurations. Thus, much attention in the 
field is devoted to the nature of the health-care sector: for example, the development of alternate 
systems such as the Singapore medical savings account model, the British NHS model, the tax- 
funded Canadian system, the Australian Medicare system and others (Kulesher and Forrestal, 
2014; Hsiao and Shaw, 2007; Blomqvist, 2011) without paying a great deal of attention to the 
specific nature of the tools deployed in each regime which give it its distinctive character. 

Conversely, but equally problematically, many studies emphasize micro-level calibrations and 
programme settings and underscore the variation that exists at this level but without reconciling 
this with the similarities which may exist in more macro-level policy parameters. These micro- 
level factors include, for example, bureaucratic routines and imperatives of specific agencies or 
individual decision-makers. They often focus on the efficacy of specific program parameters 
such as the level of subsidy, contribution rates or benefit levels in social policy schemes and often 
highlight the idiosyncratic processes underlying these choices (Asher and Bali, 2015; Hsiao and 
Shaw, 2007; Langenbrunner et al., 2009). While such micro-level studies are useful in under- 
standing how programmatic variations affect service delivery and costs, however, they err in the 
opposite direction of macro models in comparative policy studies as they are too idiosyncratic 
and ignore how policy choices at this level are constrained by choices of policy tools and objec- 
tives as well as governance arrangements and ideas. 

To move the debate beyond this overemphasis on both policy regimes and micro-level policy 
models, many current studies now focus on the specific meso-level characteristics which many 
traditional policy studies tended to neglect. This meso level is composed of policy instruments 
or techniques that give policy goals effect, especially the mixes of such tools which comprise 


Table 8.2 Policy Components 


High-Level Abstraction Programme-Level Abstraction Specific On-the- Ground Measures 


Policy Focus: Ends Policy goal Objectives Specification 
Policy Focus: Means Tools logic Policy tools Calibration 


Source: Cashore and Howlett, 2007 


104 


Primary and Secondary Relationships 


most modern policies. Such studies deal with how policy tools are assembled, sequenced and 
deployed; how such actions affect and are affected by macro-level policy change and micro-level 
policy dynamics; and the implications of these findings for future policy designs (Howlett and 
Lejano, 2013; Howlett et al., 2018). This highlights the significance of the need for a clearer 
understanding of policy mixes and how tools originate and interact in such mixes to produce 
desired, or undesirable, policy effects. 


Improving the Analysis of Policy Mixes 


Distinguishing Between Primary and Secondary or Supplementary Tools 


One issue which needs resolution in the study of policy mixes, as set out earlier, is reconciling the 
specific functionalities of policy tools with the multiple objectives they are intended to achieve. 
Even straightforward policy goals — such as health care for all — in reality comprise a vast array of 
objectives, each of which requires deployment of multiple policy tools if they are to be achieved. 
Delivering health care to all, for example, requires expanding the supply of health services which, 
in turn, requires mobilizing the resources necessary to produce the services and then deliver- 
ing them to those according to differential needs. In addition, there is the unstated objective of 
accomplishing this aim in a financially sustainable manner. Each of these efforts to produce and 
distribute health-care services in an efficient and equitable manner requires multiple tools that are 
bundled together in ways in which they engender complex patterns of interactions and outputs. 

Moreover, policy problems are often characterized by multiple dimensions, each of which 
requires a different solution. Thus, for example, what is called a “health-policy problem” 
typically consists of three interrelated sets of problems — centred on provision, financing and 
payment — with each set comprising its own distinct set of problems and possible solutions. As 
noted earlier, a health system characterized by private provision (delivery) has major implications 
for efficiency and equity which pose different obstacles to attaining the goal of universal access 
than those posed by public systems. 

Hence, it is important to note not only that different policy tools co-exist but also that they do 
so in a hierarchical fashion. That is, some tools (and resources) are “primary” in the sense that they 
are central to achieving a stated objective and area expected to be the primary instrument(s) for 
changing a targeted behaviour. Others are “secondary” in the sense that, by themselves, they are not 
expected to achieve primary policy goals but rather to play a vital role in supporting primary tools. 

The primary and secondary or “supplementary” tools used to target problems together form 
a policy “portfolio” (see Table 8.3). 

While contemporary design studies recognize that discrete policy tools are typically “bun- 
dled” in a “portfolio”, ideally working in concert to achieve stated goal, they do not help identify 
which tools have, or should have, priority in such mixes. Policy design scholars have, however, 
become increasingly interested in establishing guidelines for the formulation of more sophis- 
ticated policy designs in which complementarities across orders are maximized and conflicts 


Table 8.3 Two Orders of Policy Tools 


Order Definition 

First Order The primary substantive and procedural tools used to meet policy objectives 

Second Order Other tools that are layered to increase effectiveness or sharpen the effect of first-order 
tools 
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avoided and in identifying the various means through which mixes can be developed which are 
generally resilient and robust in the face of change (Capano and Woo, 2017). This research asks 
meaningful questions, for example, concerning the utility of redundant elements in policy mixes 
which might seem a priori to be dysfunctional or inefficient but which can, in fact, promote 
resiliency and adaptiveness (Bali et al., 2019). 

Identifying supplementary policy tools that condition or sharpen the effect of the main tool, 
however, would help present a more complete picture of the individual policy tool (Hou and 
Brewer, 2010) and the supporting institutional architecture (beyond ideas, governance styles, 
actors) that conditions its performance and is critical to effective policy design. Understanding 
such relationships would lend itself to more rigorous policy design studies and a better under- 
standing of the conditions under which policy success and failure occur (Wilder, 2017). 


Distinguishing More Clearly Between Substantive and Procedural Tools 


A second significant problem in policy mix research is the tendency among both scholars and 
practitioners to concentrate exclusively on substantive implementation tools while overlooking 
associated procedural tools (Howlett, 2019; Bali et al., 2021). Substantive tools — such as sub- 
sidy, tax, public enterprises, etc. — seek to directly target the problematic conditions or subjects’ 
behaviour while procedural policy tools are decision processes used to alter the choice or use 
of substantive tools. (Howlett, 2000). Procedural tools do not affect outcomes as directly as 
substantive tools, but they are often essential for the latter to work. Each substantive policy tool 
relies on a range of secondary complex administrative processes and procedures which are used 
to select, sequence, calibrate and deploy the tool, such as the administrative processes and activi- 
ties involved in selecting, deploying and calibrating such tools. 
Table 8.4 provides examples of such procedural tools in the health-care sector. 


Table 8.4 Procedural Tools in Health Care: Examples 


Policy Formulation Tools Accountability Tools 


National health plan Performance-based contracts 


National targets, goals and performance measures 


Multi-year strategic plan; medium-term expenditure 
frameworks and national health accounts 


Partnership Tools 
Inter-ministerial and inter-departmental committees 


Public-private task force 
Integrated budgets and accounting, co-funding 
arrangements or earmarked grants 


Information-Gathering Tools 
Inter-ministerial fact-finding commissions or task 
force, parliamentary investigations 


Periodic audits, public expenditure and performance 
reviews 
Health technology assessments 


Source: Adapted from Barbazza and Tello (2014) 
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Systems of judicial redress, procedures to combat 
corruption 
Superintendent board/board of governors 


Participation Tools 

Open meetings, public workshops, national forums, 
citizen advisory committees, citizen juries 

Satisfaction surveys 

Formal consultations in drafting legislation 


Transparency Tools 

‘Watchdog committees (facility boards, health 
authority, ombudsman, parliamentary 
committees) 

Releasing performance information to providers 
and the public 

Inspectorates, fact-finding commissions 


Primary and Secondary Relationships 


Similar to substantive tools, procedural tools, too, come in primary and secondary forms. 
That is, some procedural tools are more “primary”, such as those related to designing gover- 
nance arrangements (partnerships, hierarchies, degree and type of competition, accountability 
mechanisms, etc.) for delivering services or those which establish or change actors’ positions, 
add actors to networks or change access rules for actors. But other uses are more supportive 
or “secondary”, such as changing evaluative criteria or influencing payoff structures for actors, 
such as codes of conduct and behaviour; tools used to regulate conflict, such as tribunals or 
commissions; or those which otherwise change interaction procedures, certify certain types 
of behaviour or change supervisory relations between actors (Agranoff and McGuire, 1999). 


Summary: The Need to Revisit and Revise Common Conceptions 
of Policy Portfolios in Order to Advance Policy Studies and Practice 


In Table 8.5, we adapt Cashore and Howlett’s original six-part conception of policy elements 
(Table 8.2) to present a more comprehensive depiction of policy tools which takes into account 
the distinctions between substantive and procedural tools and their primary and secondary 
functions highlighted earlier. These distinctions enable analysts to better identify factors that 
affect a tools mix’s performance and, we argue, should form an integral part of the study of 
policy tools and policy design. 

This revision of the Cashore and Howlett model of policy elements is more robust and better 
suited to the analysis of policy mixes than the original formulation. 


Conclusion 


A key theme in the new design orientation in the policy sciences is to understand how contem- 
porary policy problems can be better addressed in heterodox contexts. This, in turn, requires a 
sophisticated understanding of the spectrum of policy tools government use and the factors that 
condition their performance. Our argument in this chapter is that existing frameworks of policy 


Table 8.5 An Improved Depiction of Policy Elements 


Macro Level Meso Level Micro Level 
Policy Ends Goals Objectives Specifications 
Aspirational ideas The policy’s stated purpose Establishing arrangements 
driving the policy and conditions for the 
chosen policy to work 
Policy Means Substantive Focus Primary and Secondary Substantive tools Substantive Calibrations 
Dominant policy Substantive tools to be deployed to Functional calibrations of 
style and tool achieve policy objectives primary and secondary 
preferences Other substantive tools that sharpen or tools 
increase the effectiveness of primary 
tools 
Procedural Focus Primary and Secondary Procedural tools Procedural Calibrations 
Dominant Decision-making processes for Size and composition 
governance ideas operationalizing the chosen of the procedural 
and arrangements substantive tools decision-making 
Supporting processes to enhance the bodies 


effectiveness of primary tools 
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tools understate the complexity of policy tools and policy mixes. These include a failure to 
incorporate into existing models (1) the multiple objectives (requiring different types of govern- 
ing resources) policy tools face, (2) the hierarchical relationships among different tools in a mix 
and the fact that (3) procedural tools play a key role in all policy designs. 

To address these challenges, we adapted Cashore and Howlett’s (2007) model of policy 
decomposition to distinguish between primary and secondary tools at both the substantive and 
procedural levels. The framework presented in Table 8.5 is a useful heuristic or diagnostic tool 
to better understand the institutional architecture that supports a policy sector and conditions its 
performance. The additional nuance and understanding such a framework brings allows analysts 
to gain a deeper understanding of the tools used and the institutional architecture that condi- 
tions their performance than do current policy efforts which fail to account for the primary and 
secondary nature of the procedural and substantive tools arrayed in such mixes. 
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In examining the policy sciences literature on policy tools and instruments, we find that the 
focus has mostly been on implementation tools. So this chapter presents several types of policy 
tools used by governments at the agenda-setting stage of the policy cycle. The agenda-setting 
stage is not a process driven entirely exogenously and thus outside of the purview of government 
activity; rather, governments control or attempt to control issue prominence and agenda-entry 
patterns in an effort to manage or direct their own policy timetables and agendas. This chapter 
adds to Christopher Hood’s taxonomy of available toolsets of resources available to governments 
by identifying examples of agenda-setting tools available to governments. 


Introduction: Non-Implementation Tools and Public Policymaking 


The literature in the policy sciences on policy tools or instruments historically has focused 
almost exclusively on implementation tools (Hood 1986; Salamon 2002; Hood and Margetts 
2007; Howlett 2011). However, each of the stages of the policy process is also subject to gov- 
ernment action (Wu et al 2010), and distinct sets of policy tools can be identified in each of the 
non-implementation stages of policymaking. 

This chapter addresses the situation with respect to agenda-setting or the stage in which 
policy issues and problems are first brought to the attention of policymakers and included as 
subjects of possible government policy-making activities (Cobb and Elder 1972; Kingdon 
1984). 

While many accounts of policymaking assume agenda-setting to be a process driven entirely 
exogenously and thus outside of the purview of government activity, detailed analyses of agenda- 
setting issue formation and behaviour have not supported this view (Cobb et al. 1976; Howlett 
1997; Baumgartner and Jones 1993). Rather, these studies have all identified common practices 
whereby governments control or attempt to control issue prominence and agenda-entry pat- 
terns in an effort to manage or direct their own policy timetables and agendas (Rochefort and 
Cobb 1993). 

This chapter examines this literature and focuses attention on several tools commonly 
employed in these efforts, ranging from funding for and regulation of stakeholder and interest 
group formation to information and public opinion formation through government advertis- 
ing to media control or influence through freedom of information and privacy legislation. The 
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chapter highlights the significance of these efforts and the utility of using frameworks and tax- 
onomies developed in the study of implementation tools for their analysis. 


What Are Policy Tools? 


Policy alternatives are composed of different sets or combinations of the policy elements 
described earlier. Policy instruments are the techniques or means through which states 
attempt to attain their goals. They are the subject of deliberation and activity at all stages of 
the policy process as they affect both the agenda-setting and policy formulation processes as 
well as being the subject of decision-making policy implementation and evaluation (Howlett 
2005). 

They have a special place in the consideration and study of policy design because, taken 
together, they comprise the contents of the toolbox from which governments must choose in 
building or creating public policies. Policy design elevates the analysis and practice of policy 
instrument choice — specifically tools for policy implementation — to a central focus of study, 
making their understanding and analysis a key design concern (Salamon 1981; Linder and Peters 
1990). Instrument choice from this perspective in a sense, is public policymaking, and under- 
standing and analyzing potential instrument choices involved in implementation activity is pol- 
icy design. The role of a handbook in policy design thus is one of assisting “in constructing an 
inventory of potential public capabilities and resources that might be pertinent in any problem- 
solving situation” (Anderson 1977, p. 122). 

It is important to repeat, however, that policy instruments exist at all stages of the pol- 
icy process — with specific tools such as stakeholder consultations and government reviews 
intricately linked to agenda-setting activities, those like legislative rules and norms linked to 
decision-making behaviour and outcomes and others linked to policy evaluation, such as the use 
of ex-post, or after-the-fact, cost-benefit analyses (see Figure 9.1). 

Although policy instruments appear in all stages of the policy process, those affecting the 
agenda-setting, decision-making and evaluation stages of the policy process, while very signifi- 
cant and important in public management (Wu et al. 2010), are less so with respect to policy 
design activities. This is because policy design largely takes place at the formulation stage of 
the policy cycle and deals with plans for the implementation stage. Thus, the key sets of policy 
instruments of concern to policy designers are those linked to policy implementation in the first 
instance and to policy formulation in the second. In the first category, we would find examples 
of many well-known governing tools such as public enterprises and regulatory agencies, which 
are expected to alter or affect the delivery of goods and services to the public and government 
(Salamon 2002), while in the second, we would find instruments such as regulatory impact or 
environmental impact appraisals, which are designed to alter and affect some aspect of the nature 
of policy deliberations and the consideration and assessment of alternatives (Turnpenny et al. 
2009). 

As pointed out in earlier chapters, policy tools are “multi-purpose”, since, for example, 
regulation can appear in the implementation activities of several governance modes while some 
tools, like impact assessments, can also appear in several stages of the cycle. However, a regu- 
lation appearing within the implementation phase of a network mode of governance which 
mandates information disclosure, for example, serves a different purpose than a regulation 
found in a market mode which limits a firm to ownership of only a specific percentage of an 
industry. Similarly, consultations which take place in the agenda-setting stage of the policy 
process have a different purpose and effect than those which take place after a decision has 
been made. 
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Figure 9.1 The Full Range of Policy Instruments 


What Is an Agenda-Setting Tool? 


What Is Agenda-Setting 


e.g. Trade Surveys 


e.g. Regulatory 
Impact 
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e.g. Multi-Criteria 
Analyses Focusing 
on Employment 
and Income Effects 


e.g. Cost-of-Service 
Price Regulations 
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As Cobb and Elder put it in their early studies of the subject in the United States: 


[P]re-political, or at least pre-decisional processes often play the most critical role in 
determining what issues and alternatives are to be considered by the polity and the 
probable choices that will be made. What happens in the decision-making councils of 


the formal institutions of government may do little more than recognize, document 


and legalize, if not legitimize, the momentary results of a continuing struggle of forces 


in the larger social matrix. . 


. . From this perspective, the critical question becomes, 


how does an issue or a demand become or fail to become the focus of concern and 


interest within a polity? 


(Cobb and Elder 1972) 


The question of how a problem comes to be interpreted as a public problem requiring gov- 
ernment action raises deeper questions about the nature of human knowledge and the social 
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construction of that knowledge, and the policy sciences took many years to evolve a position or 
theory on the nature of social problems. 

It has long been generally agreed, however, that a variety of political (Castles and McKinlay 
1979; Castles 1982; Hibbs 1977; King 1981; von Beyme 1984), epistemological (Hilgartner 
and Bosk 1981; Holzner and Marx 1979; Rochefort and Cobb 1993) and ideological (Edel- 
man 1988; Stark 1992; Fischer and Forester 1993; Stone 1988, 1989) factors in addition to the 
unfolding of basic socio-economic processes can affect which social problems gain access to 
the formal policy agenda of government. This conceptual agreement on the basic outline of 
significant policy determinants, far from representing the culmination of analysis in this area of 
public policymaking, merely sets out the general types of variables which analysts should take 
into account when investigating the subject. In this same vein, a wide variety of analysts from 
a disparate range of fields have endorsed a model of agenda-setting first put forward by John 
Kingdon in his 1984 work on the operation of the US federal legislative system. His model 
deals with the question of state and non-state influences on agenda-setting by focusing on the 
role played by policy entrepreneurs both inside and outside of government in taking advantage 
of agenda-setting opportunities — policy windows — to move items onto formal government 
agendas. It suggests that the characteristics of issues — the problem stream — combine with the 
characteristics of political institutions and circumstances — the politics stream — and the develop- 
ment of policy solutions — the policy stream — in a fashion which can lead to the opening and 
closing of windows of opportunity for agenda entrance. Such opportunities can be seized on 
or not, as the case may be, by policy entrepreneurs who are able to recognize and act on them. 

Kingdon’s model of agenda-setting represents the orthodoxy in policy studies. Among other 
subjects, it has been used to describe and assess the nature of US foreign policymaking (Woods 
and Peake 1998); the politics of privatization in Britain, France and Germany (Zahariadis 1995; 
Zahariadis and Allen 1995); the nature of US domestic anti-drug policy (Sharp 1994); the col- 
laborative behaviour of business and environmental groups in certain anti-pollution initiatives in 
the US and Europe (Lober 1997); and the overall nature of the reform process in Eastern Europe 
(Keeler 1993). It leaves open, however, the key questions of which policy actors are involved in 
agenda-setting and the manner in which the highlighted variables combine to affect the actions 
of these actors. 

Beginning from the observation that the process of agenda-setting involves discussion, debate 
and persuasion among policymakers, who present a variety of evidence and argument in sup- 
port of their position (Majone 1989), scholars such as Frank Baumgartner and Bryan Jones 
developed models of agenda-setting which focused on the significance of policy subsystems 
(Baumgartner and Jones 1991, 1993, 1994). That is, the two key actors in agenda-setting are 
governments and the “public”, and the relationship set out between them was one in which the 
government responded to public concern. Both sets of actors are often assumed to be driven 
largely by self-interest: the government by its political wing and politicians’ interest in reelection 
in a democratic polity and the public by rational calculations of what individuals felt could be 
accomplished as a result of their devotion of scarce political resources to particular problems. 
The key element in the process of agenda-setting, in this view, revolves around the ability of state 
and societal actors to control the interpretation of a problem and thus the manner in which it is 
conceived and discussed (Howlett and Ramesh 1995). 

Many other models similarly focused on the nature of the actors involved in the process and 
the general characterization of agenda-setting “styles” in terms of typical relationships of power 
and influence existing between significant actors (Cobb, Ross and Ross 1976; Rochefort and 
Cobb 1993). These studies tended to support the view that agenda-setting in democratic states 
is largely a matter of governments responding to social pressures and focused their attention on 
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how the activities of interest groups could facilitate this process (Bali and Halpin 2021). Other 
studies, however, pointed to a much larger role played by government agencies and a variety 
of “boundary-spanning” organizations, such as the media, in blocking, filtering or otherwise 
affecting the development of public concerns, undermining the notion that agenda-setting was 
a relatively simple, one-way, transmission process (Downs 1972; Howlett 1997, 1996; Hogwood 
1992). 

That is, there is as much evidence that the government agenda drives the public as there is 
for the reverse situation. 


What Is an Agenda-Setting Tool? 


Implementation tools are the policy instruments most often studied. They affect either the 
content or processes of policy implementation: that is, which alter the way goods and services 
are delivered to the public or the manner in which such implementation processes take place 
(Howlett 2000). Many of the distinctions and categorizations developed to examine implemen- 
tation tools remain useful in examining other types of tools as well. 

One common category of implementation instrument thus, for example, proposes to alter 
the actual substance of the kinds of day-to-day production, distribution and consumption activi- 
ties carried out in society while the other focuses upon altering political or policy behaviour in 
the process of the articulation of implementation goals and means. “Substantive” implementa- 
tion instruments are those used to directly affect the production, distribution and consumption 
of goods and services in society while “procedural” implementation instruments accomplish the 
second purpose (Ostrom 1986; Howlett 2000, 2005). 

That is, at their most basic level, all government tools fall into these two types (substantive or 
procedural), depending on their general goal orientation. Substantive instruments are expected 
to alter some aspect of the production, distribution and delivery of goods and services in society, 
broadly conceived to include both mundane goods and services (like school lunches) as well as a 
range of vices and virtues, from crude vices (such as gambling or illicit drug use) to more com- 
mon individual virtues (such as charitable giving or volunteer work with the physically chal- 
lenged) to the attainment of sublime collective goals (like peace and security, sustainability and 
well-being) and are found at the implementation stage of the policy cycle, although, of course, 
discussed and decided upon elsewhere. 

Procedural tools, on the other hand, affect production, consumption and distribution pro- 
cesses only indirectly, instead affecting the behaviour of actors involved in policymaking. These 
actors are arrayed in policy networks which comprise very simple arrangements of nodes (actors) 
and links (relationships) but can result in very complex structures and interaction patterns. Policy 
networks include sets of formal institutional and informal relational linkages between govern- 
mental and other policy actors, which are typically structured around shared beliefs and interests 
in public policymaking and implementation. In order to pursue their preferred policy initiatives, 
governments must interact with other state and non-state actors who might possess diverging 
interests (Leik 1992). They use procedural tools to alter the behaviour of policy network mem- 
bers involved in policy-making processes. They are only tangentially related to productive or 
consumptive behaviour, if at all, and figure more clearly at stages such as agenda-setting. 


Types of Government Agenda-Setting Tools 


This chapter presents the different types of policy tools governments can use at the agenda- 
setting stage of the policy cycle. By examining the toolsets available to the government at this 
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Table 9.1 Examples of Agenda-Setting Tools by Resource Used 


Nodality Authority Treasure Organization 
Government Communications Sunset clauses Charitable deductions Scenario planning 
Issue management Periodic review Strategic foresight 


stage, we can better study and understand how governments control or attempt to control issue 
prominence while trying to manage or direct their own policy timetables and agendas. 

Christopher Hood presented a toolset that governments use to govern which he called the 
NATO scheme and argued that governments tend to possess four basic resources by virtue 
of being government: nodality, authority, treasure and organization (Hood 1986). Nodality 
denotes the use of government information resources to influence and direct policy actions 
through the provision or withholding of “information” or “knowledge” from societal actors. 
Authority denotes the possession of legal or official power: i.e., the power officially to demand, 
forbid, guarantee, adjudicate. Treasure denotes the possession of a stock of moneys or “fungible 
chattels”. Organization denotes the possession of a stock of people with whatever skills they 
may have (soldiers, workers, bureaucrats), land, buildings, materials, computers and equipment, 
somehow arranged (Hood and Margetts 2007). 

Hood’s NATO taxonomy can be used to identify examples of the different agenda-setting tools 
available to governments. Table 9.1 lists several specific types of agenda-setting tools available. 


Specific Types of Procedural Agenda-Setting Tools 


Nodality Tools: Government Communications and Issue Management 


Government communications are the “sermons” in the “carrots, sticks, organizations and ser- 
mons” formulation of basic policy instrument types. Evert Vedung defines these “sermons” as 
“Efforts to use the knowledge and data available to governments to influence consumer and 
producer behaviour in a direction consistent with government aims and wishes” and/or “gather 
information in order to further their aims and ambitions” (Vedung and van der Doelen 1998). 
This definition, while useful, is limited in that it conceals or elides several dimensions of infor- 
mation tool use and the general purposes to which they can be put. 

Two dimensions of government communications activities in particular are often incor- 
rectly juxtaposed in the literature on the subject. First, whether the communication activities are 
intended to serve as devices primarily oriented towards the manipulation of policy actors (Saward 
1992; Edelman 1988) or social and economic ones (Hornik 1989; Jahn et al. 2005) and second, 
which stages of the production process or policy cycle different communication tools focus on 
(Howlett 2009). Both missing dimensions require further elaboration in order to develop a work- 
able definition and classification of communication tools for comparative purposes. 

With respect to the first “substantive” dimension, much existing literature focuses very much 
on the manipulation of the behaviour of economic actors — namely, consumers and producers — 
to the neglect of the effects such tools can and do have on other kinds of policy and policy 
network actors and activities. With respect to the second “procedural” concern, many stud- 
ies focus exclusively on the role of government communications as part of the agenda-setting 
process in government (Mikenberg 2001; Sulitzeanu-Kenan 2007), on its role in policy imple- 
mentation (Salmon 1989a, 1989b) or on their effect on consumption activities and actors versus 
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those involving productive or distributive activities. These are quite different roles and functions 
within the policy and production processes, however, and should be carefully distinguished 
from each other in order to understand the links and linkages that exist between government 
communication strategies and activities and policy outcomes such as accountability and policy 
efficacy and in order to assess any trends or directions in the use of these instruments, either 
cross-sectorally, cross-nationally or, over time, both. 


Authority-Based Tools: Sunset Clauses and Periodic Review 


A sunset clause is a legal provision that provides for the expiry of a law or part of a law at a later date. 
A sunset clause does not aim at continuity; rather it “sets the sun” on a provision or entire statute 
on a specific date unless there are substantial reasons to believe that the former should be extended 
for a determined period (Ranchordas 2014). Unlike most laws, the continued validity of legislation 
subject to a sunset clause is contingent on some future action by the legislature (Ip 2014). 

A sunset clause can be used as an agenda-setting tool as it can set an expiration date to a legal 
provision at the outset. Within Hood’s NATO taxonomy, sunset clauses and periodic reviews 
can be seen as “authority” resources used by the government to set the agenda as they can use 
their power to introduce a policy when needed and mandate the termination date ahead of time 
based on a periodic review of its usefulness. Overcoming legislative inertia can be seen as one 
of the major rationales for sunset clauses as they change the default position from permanent 
to temporary. A sunset clause provides enough leeway to extend the provision at a later date if 
necessary after a periodic review has taken place. But, more importantly, it sets a policy termina- 
tion date at the outset, which can otherwise be a difficult thing to do as there can be resistance 
to termination (Bardach 1976; deLeon 1978, 1983; Kirkpatrick et al. 1999; Geva-May 2004) by 
different groups along with legislative inertia. 

According to Veit and Jantz (2013), a sunset clause has two main characteristics. It creates a 
“threat of termination” for laws by shifting the burden of proof from those who would terminate 
a policy program to those who would renew it, and it requires a periodic review and evalua- 
tion of the effectiveness and efficiency of government functions and program. The termination 
or renewal should only occur after an ex-post evaluation has been completed so it can verify 
whether the objective was achieved or if the provision should be renewed. A periodic review 
and evaluation means that sunset clauses can serve to provide a broader legislative perspective 
and can foster evaluation activities and policy learning. 

Veit and Jantz (2013), in their empirical study of the application of sunsetting in four countries — 
the United States, Australia, Switzerland and Germany — conclude that implementation of sun- 
setting is a difficult task. They write that on the one hand, international experiences show that 
general sunset regimes often fail because of limited time and resources for sunset reviews, a lack of 
acceptance and unrealistic expectations with regard to the effects of sunsetting. On the other hand, 
sunset clauses are an established feature of the regulatory toolbox — they are used to bring about 
political compromise, to strengthen parliamentary control of regulatory activities in government 
agencies or to foster better regulation and evidence-based policymaking. So sunset clauses and 
periodic reviews can be used as tools at the agenda-setting phase to bring about specific policies 
and programs for a limited time and give the discretion to renew or terminate it as necessary. 


Treasure-Based Tools: Interest Group and NGO Funding 


Governments can use policy tools that give an economic benefit to a favored group, not just 
by appropriating money but also by exempting an event or a transaction from taxation. This 
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way, such tools can allocate benefits and have several advantages as it will not further burden the 
government’s budget and can also reduce the apparent burden of taxation on individuals. But 
economists have been quick to point out that the economic effect of such deductions can be 
equivalent to an appropriation of public funds, both from the point of view of the government, 
which is deprived of that portion of receipts from taxation, and from the point of view of the 
taxpayer, who is allowed to retain funds that would otherwise be taxed (Yarmolinsky 2000). 

Charitable deductions permit certain taxpayers to deduct their contributions to charitable, 
educational and scientific organizations in calculating their taxable income. From the perspec- 
tive of the government, allowing for charitable deductions can be seen as a carrot to incentivize 
people to give to certain kinds of charities that may have positive externalities. As noted by 
Brody and Cordes (2006) and Cordes (2011), the most visible way in which the federal govern- 
ment acts as a benefactor of non-profit sector is by allowing individuals and corporations an 
income tax deduction for the value of their charitable contributions. These deductions can be 
viewed as providing an important economic incentive for private donors to provide financial 
support to a wide range of philanthropic enterprises. It can also have the direct effect of freeing 
the government from having to perform the same service, which can thereby be seen as the gov- 
ernment subsidizing itself. The public service performed by the contribution may be something 
that government could not have done, given the lack of either the means or the will to do it, 
and could remain untended if a charitable initiative did not intervene. 

Within Hood’s NATO taxonomy, charitable deductions can be seen as a “treasure” resource, 
used by the government to set the agenda by using its authority while deciding what kinds of 
activities should be indirectly supported by giving tax-exempt status and how citizens can be 
incentivized to support certain kinds of programs that have positive externalities. They forgo 
taxes to their treasury in exchange for some activity they think is good and would like to 
promote. 


Organization-Based Tools: Scenario Planning and Strategic Foresight 


Scenario planning is a policy tool used to plan for uncertain times in the future. It is a process 
of positing several informed, plausible and imagined alternative future environments in which 
decisions about the future may be played out for the purpose of changing current thinking, 
improving decision-making, enhancing human and organization learning and improving per- 
formance (Chermack 2005). Within Hood’s (1986) NATO taxonomy, scenario planning can be 
seen as an “organization” resource used by the government to set the agenda for planning for the 
future. The government can used its resources such as people, information and materials to study 
various scenarios for the future and how it can better manage and prepare for them. 

Scenario planning has become an important agenda-setting tool in recent years. Given the 
uncertainties of public policymaking, it has the potential to prepare and better manage complex 
decisions and spot early warning signals about future problems. It can also be used to identify 
and manage conflicts and to try to find common ground for future action when there are diverg- 
ing societal interests and values. As an agenda-setting tool, it is important as it can first be used 
as a policy risk-free space to visualize, rehearse and test the acceptability of different strategies 
without being implicated by the actual constraints of day-to-day policymaking (Volkery and 
Ribeiro 2009). 

Even though people have been interested in the future and have used scenarios indirectly 
to explore it, scenario planning as a strategic planning tool is firmly rooted in the military 
and has been employed by military strategists throughout history. Modern-day scenario tech- 
niques, however, only emerged in the post-war period in the 1960s with the emergence of 
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two geographical centers in the USA and France (Bradfield et al. 2005). Scenario planning is 
still not used extensively by governments and is usually executed in a rather ad hoc and iso- 
lated manner. Volkery and Ribeiro (2009), in their review of evaluative scenario literature in 
public policymaking, find that it is mostly geared towards indirect decision support in the early 
phases of policymaking such as agenda-setting and issue framing but not in later phases such 
as policy design and implementation. They conclude that political and institutional context 
factors need to be treated with greater care in the future as making decisions under conditions 
of deep uncertainty requires not only rigorous analysis but also political will and more stable 
institutional settings and organizational capacities to build up trust and experience with adap- 
tive, flexible process formats. 

Future forecasting and strategic foresight are agenda-setting tools used to help decision- 
makers develop future-oriented policies in uncertain times (Konnola et al. 2011; Georghiou 
and Harper 2011). Habegger (2010) differentiates between the two, stating that future forecast- 
ing tries to capture and anticipate potential future developments and to generate visions of how 
society evolves and what policy options are available to shape a desired future, while strategic 
foresight can be seen as a deliberate attempt to broaden the boundaries of perception and to 
expand the awareness of emerging issues and situations. It aims to support strategic thinking and 
decision-making by developing a range of possible ways the future could unfold. Like scenario 
planning, both future forecasting and strategic foresight fall under the “organization” resource 
in Hood’s NATO taxonomy as the government uses its available organized resources to plan for 
uncertain times in the future. 

Various authors (Leigh 2003; Calof and Smith 2010; Habegger 2010) have identified many 
different factors that lead to government-led foresight success. Leigh identified five ways in 
which strategic foresight teams can contribute to more innovative government: anticipating 
emerging issues, identifying unanticipated consequences, getting a sense of the “big picture”, 
drawing a wide range of information sources and involving the public. Based on experiences 
from the United Kingdom, Singapore and the Netherlands, Habegger includes having a scien- 
tific edge in terms of specific foresight methods and processes, allowing for innovation, fostering 
iterative interactions between stakeholders and obtaining the trust and support of top bureau- 
crats to support the idea of exploring futures that may be quite different from the present condi- 
tions as elements of successful foresight exercises. 

At the agenda-setting phase, future forecasting and strategic foresight can also be used to 
inform policy by enhancing the knowledge base for thinking about and designing policies and 
can even help in identifying current policy gaps. In many cases, it can be used by policymakers 
as a signaling device to show the public that scientific rationale and planning are used in making 
policies. 


Conclusion: Agenda-Setting Tools and Policy Instrument Studies 


As Linder and Peters, Davis Bobrow, Peter May, Patricia Ingraham, Christopher Hood, Renate 
Mayntz and the other pioneers of policy design research in the 1980s and 1990s argued, like 
other kinds of design activities in manufacturing and construction, policy design involves three 
fundamental aspects: (1) knowledge of the basic building blocks or materials with which actors 
must work in constructing a (policy) object, (2) the elaboration of a set of principles regarding 
how these materials should be combined in that construction and (3) understanding the process 
by which a design becomes translated into reality. In a policy context, this means understand- 
ing the kinds of implementation tools governments have at their disposal in attempting to alter 
some aspect of society and societal behaviour, elaborating a set of principles concerning which 
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instruments should be used in which circumstances and understanding the nuances of policy 
formulation and implementation processes in government. 

Establishing the nature of the policy “space” is a crucial activity in analyzing policy tool 
choices (May 1981). Generally speaking, policymakers should “consider the range of feasible” 
options possible in a given circumstance and package these into sets of “competing strategies” to 
achieve policy goals (May 1981, pp. 236 and 238). As David Weimer (1992) has argued, “Instru- 
ments, alone or in combination, must be crafted to fit particular substantive, organizational and 
political contexts” (p. 373). 

This is just as true of agenda-setting tools as any other. While this chapter has gone some way 
towards describing and classifying agenda-setting tools and instruments, a much larger research 
agenda remains to describe the patterns of, and reasons for, their use. 
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FORMAL AGENDA-SETTING 
GOVERNMENT TOOLS 


Periodic Evaluations and Reviews 


Valerie Pattyn and Pirmin Bundi 


The dominant focus in agenda-setting scholarship on external factors driving governments’ 
agenda can make us overlook that governments also have many formal tools at their disposal to 
steer and manage their own agenda and timetables. In this chapter, we study the agenda-setting 
potential of three such procedural tools, which have thus far received little attention in prior 
research: periodic policy reviews, spending reviews, and sunset reviews. We discuss their major 
attributes, their underpinning rational assumptions, and the way they have been institutional- 
ized and routinized in countries’ evaluation systems and reflect on the conditions that foster or 
jeopardize their agenda-setting impact. Throughout the chapter, we draw on different empirical 
examples from various countries. While these tools are very often strongly institutionalized, the 
actual impact on agenda-setting should generally not be overestimated. 


Introduction 


The dominant focus in agenda-setting scholarship on external factors driving governments’ 
agenda can make us overlook that governments also have many formal tools at their disposal 
to steer and manage their own agenda and timetables (Rochefort and Cobb, 1993; Howlett 
and Shivakoti, 2014). Such in-house government tools get much less attention than exog- 
enous agenda-setting forces, even though they may be as influential as social pressures or 
pushes from interest groups on governments’ priorities and timetables (Baumgartner and 
Jones, 1993). 

In recent decades, governments worldwide increasingly invested in the institutionalization 
of these tools, with the intention of controlling issue prominence. In this chapter, we review 
the agenda-setting potential and influence of a particular group of evaluation-related tools gov- 
ernments often resort to in the agenda-setting stage. Policy evaluations, which we define as 
‘structured and objective assessments of the content, implementation, or impact of policies or 
programmes’ (OECD, 2020), are typically conceived in a retrospective ex-post way only and as 
the concluding stage of the policy process (Vedung, 1997). 

However, particular evaluation tools also play an important analytical role in setting up and 
managing the agendas of governments. This contribution discusses the agenda-setting role of 
three in-house tools: budget reviews, policy reviews, and sunset reviews. While each of these 
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tools looks backward, they also generate and block policy options for the future. It is this hybrid 
nature that makes them particularly interesting to study. 

Policy analysis literature and policy tool scholarship has thus far paid little attention to them. 
In this chapter, we address this void. In the first section, we concisely discuss what character- 
izes them as a particular group of policy tools and with which rationale they are generally 
introduced by governments. Next, we review the potential of each of the three tools in theory. 
Thereafter, we focus on the actual value of these tools as empirically observed in practice and 
discuss several drivers and impediments for impact. We conclude with a general discussion of 
our findings. 


Evaluation Tools With Agenda-Setting Potential: 
Classification and Rationale 


A full understanding of the role of evaluation tools in the agenda-setting stage requires a com- 
prehensive approach to policy instruments, in line with the more recent shift in policy design 
studies since the late 1990s (Howlett, 2019a). Policy evaluations or reviews are not always con- 
sistently classified in policy analytical literature, however, and in the many approaches to policy 
instruments circulating (see e.g., Acciai and Capano, 2021 for an overview). Radin (2013), for 
instance, considers evaluation tools as an exponent of the broader family of analytical tools. 
Schick (1977), along the same line of thinking, approaches them as a type of analycentric or 
decision support tools. 

Relying on Howlett (2019a) and drawing on the extended NATO typology (Hood, 1983), 
evaluation tools can be conceived as well as procedural policy instruments. Procedural policy 
tools, other than substantive tools, only indirectly impact the production, consumption and 
distribution processes of target groups through affecting the behavior of actors involved in poli- 
cymaking themselves (Howlett, 2019a: 56). More in particular, the type of evaluation tools at 
stake in this chapter has the potential to (re)structure the policy process by altering the num- 
ber of policy options in the policy process or the calculations of costs and benefits of different 
options possible (cfr. Bressers and Klok, 1988; Howlett, 2000). As a result, they can influence the 
substantive outcome of implementation activities. Common to the tools discussed is that they 
are formally embedded in government legislation or regulation. They all require the (re)consid- 
eration of policy priorities at particular time intervals. 

Further drawing on this typology, and as propagated in Howlett and Shivakoti (2014), the 
evaluation tools can be conceived as cases of authoritative policy instruments. Authority implies 
that governing bodies of the public sector, such as government agencies or parliaments, possess 
the power to demand evaluations (Bundi, 2016; Pattyn et al., 2019; Bali and Halpin, 2021). 
They mandate certain procedural requirements in the policy-making process to ensure that poli- 
cies pass the test of effectiveness, efficiency, or other evaluation criteria often stipulated in law 
or regulations. Interestingly, the target group of such authoritative instruments are policymakers 
themselves. Whether the evaluation tools are indeed effective and implemented as mandated 
will thus also largely depend on how policymakers conceive the legitimacy and relevance of this 
mandate. 

To be sure, our chapter centers on in-house instruments. The in-house qualification in the 
first place concerns the actor commissioning their production, with ‘in-house’ usually referring 
to government ministries (1.e., for policy reviews) or the cabinet committee (1.e., for spending 
reviews). Typical for these tools, which by nature rely on scientific procedures, is the frequent 
involvement of research-oriented parties in the production of the evidence, however, such as 
universities, government research institutions or consultants. 
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Finally, the type of instruments at stake is of primary relevance for the agenda-setting stage. 
As hinted at before, such evaluation tools can in fact be used throughout the entire policy cycle, 
including when a policy is still in operation or when a policy has already been concluded. Also, 
the type of evidence these tools produce tends to be of a mixed nature, with evidence yielding 
suggestions of ‘where to go’, which is typically connected with agenda-setting but at times also 
pointing at routes about ‘how to get there’, the latter being associated with policy formulation 
(Turnpenny et al., 2015: 4). In sum, the classification of evaluation tools as particular policy 
instruments serves in the first place as an analytical heuristic and should be treated with care. 

While the commitment to particular policy tools will be contingent on the interplay between 
macro, meso and micro factors (Linder and Peters, 1989; Peters, 2000), and while each of the 
tools discussed has a particular rationale, the introduction of evaluation instruments such as 
policy reviews, spending reviews and sunset reviews is to be mainly situated with the introduc- 
tion of New Public Management (NPM) in the late 1970s. This neoliberal wave came with the 
incorporation of private sector principles in the public sector and a heavy emphasis on value 
for money and efficiency (Vedung, 2010; De Francesco and Pattyn, 2021). One of the central 
principles of NPM is output-based management and control by measuring government perfor- 
mance (Vedung, 2010). As a result, with evaluation being a key tool for measuring outcomes and 
impacts and responding to the need for improved accountability, NPM served as a major driver 
for the introduction of evaluation tools in the public sector (Pattyn et al., 2018). 

Besides NPM, ‘evidence-based’ policy or ‘evidence-informed’ policy as a normative prin- 
ciple and the ‘what works’ discourse (Vedung, 2010) was a major impetus for the dissemination 
of evaluations and policy reviews, also for the purpose of agenda-setting. For the 2008 financial 
crisis, for instance, evaluation tools were seen as having the potential to help governments mak- 
ing decisions what to cut and to find new ways of addressing public needs and delivering services 
(Stame, 2012; Mavrot and Pattyn, 2022). More in general, evidence-informed policy is seen as 
a way to reduce non-design spaces by prioritizing policy decisions that are justified by theoreti- 
cally informed empirical analysis (Howlett, 2019a: 64). 


Agenda-Setting Potential of Procedural Authoritative Evaluation Tools 


Budget Reviews 


Given their backward-looking and forward-looking approach, budget reviews have been por- 
trayed as tools breathing the spirit of integrated policy analysis or policy design (van Nispen 
and de Jong, 2017). Various modes and classifications of budget reviews exist, with spending 
reviews and expenditure reviews commonly seen as the main types, although the distinction is 
not always consistently applied in practice. While expenditure reviews may result in suggesting 
proposals for extra funding, spending reviews are focused on identifying potential savings in rela- 
tion to the baseline and examining the consequences of alternative lower levels of funding. They 
can be further subdivided in one-off and semi-permanent proposals for smart savings based on 
the assessment of the effectiveness and efficiency of policy measures (i.e., functional reviews) or 
the timeliness of the objectives (i.e., strategic reviews) (van Nispen and de Jong, 2017). Budget 
reviews are mainly coordinated by a cabinet committee under the auspices of either the Prime 
Minister or the Minister of Finance. Line ministries are often in charge of drafting reports 
of their particular ministry and are consequently actively engaged in the review process (van 
Nispen, 2016). 

Especially since the 2008 financial crisis, budget reviews were reinvigorated in popularity as 
strategic tools that could help countries identifying savings in a ‘smart’ way (OECD, 2011; van 
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Nispen, 2016). As reported by the OECD in 2019, all but six OECD countries at the time used 
budget reviews to some extent and in some format. Empirically, much variation exists in the 
frequency and way in which budget reviews are conducted across countries, and it would exceed 
the scope of this chapter to elaborate on this in detail. Countries such as the Netherlands and 
Germany, for instance, organize spending reviews of selected spending programmes each year 
within a limited area. Other countries, such as the UK and Ireland, prioritize comprehensive 
spending reviews, which take place at specific time intervals, often in response to the election 
of a new government or between elections — for instance, in reaction to tough fiscal pressures 
(OECD, 2019a). 

Comprehensive reviews, as the term itself reveals, are broader in scope and are focused 
on the whole of government. They are to large extent similar to zero-based budgeting (van 
Nispen, 2016). Also the COVID-19 pandemic seems to have triggered the organization of 
(comprehensive) spending reviews, which was the case in Flanders (Belgium), for instance, 
where such review was launched for the first time in 2021. Spending review practices are also 
rapidly evolving, with innovative new types being adopted in recent years. Noteworthy, for 
instance, is the 2017 introduction of the Public Values Framework in the UK, inspired by 
Mootre’s public value theory (1995). The framework seeks to define everything that a public 
body should be doing to maximize the likelihood of delivering optimal value from the fund- 
ing it receives. It uses a set of criteria to assess how likely it is that value is being maximized 
by the activities that are taking place. These criteria revolve around four pillars relating to 
overarching goals the public body aims to achieve and the progress made towards it, the 
public body’s financial management, the involvement of citizens and users and the extent to 
which system capacity is developed (HM Treasury, 2019: 4). Also, other countries such as the 
Netherlands recently piloted a public values scan (Publieke Waarde Scan), modeled along the 
UK framework. 

Irrespective the particular type, budget reviews are a tool par excellence to give the govern- 
ment improved control over the level of aggregate expenditure and have potential to affect the 
policy agenda. Governments can use the tool to cut less effective and efficient spending, which 
enables them to safeguard key spending priorities or to reprioritize expenditure that does not 
run counter to higher-level government policy objectives and wider fiscal objectives. As a result, 
they can be seen as a tool to block spending decisions that are driven by narrow, near-term 
interests. 

Budget reviews from this perspective contrast with the normal focus in the budget, which 
tends to be characterized by (competing) demands for incremental increases (OECD, 2019a: 
113). For reviews, the budget cycle serves as the most important platform for deciding on the 
outcomes of the reviews and incorporating potential savings. Quite evidently, this is a strong 
political process rather than an economic one (van Nispen and de Jong, 2017: 158) as govern- 
ments have much discretion in deciding whether the results generated by the reviews will be 
used and which of the proposed scenarios for savings or reprioritization are adopted. With 
spending reviews not taking current funding as a given, they de facto also shift the burden of 
proof to those who want to continue the current program (van Nispen, 2016: 482). Budget 
reviews are thus an inherent political instrument, which is no different than any other evaluation 
tool (Mavrot and Pattyn, 2022). 


Policy Reviews 


Although the term ‘policy reviews’ is frequently used to refer to evaluations of single programs 
or policy measures, the very label is, in principle, reserved for periodic and systematic reviews 
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of policies. Policy reviews are best seen as a synthesis of existing program evaluations, which — 
provided that such evaluations are available — do not require additional research or fieldwork. 

Just like budget reviews, policy reviews are promoted for their ability to deliver policy-relevant 
information on the effectiveness and efficiency of current policies and can impact governments’ 
decision to proceed on the same basis, to implement changes or to terminate policies. Given 
their synthetic nature, policy reviews also hold the potential to identify overlap and wanted or 
unwanted side effects between different policy measures. They can, as a result, contribute to the 
resilience and robustness of policies by making recommendations to make different policy mea- 
sures more coherent and consistent and to bring them in syne with each other (Howlett, 2019b). 

Although systematic reviews are in vogue, generally speaking, not many countries have 
developed procedures and guidelines for policy reviews, at least to our knowledge. Scholarly 
works discussing policy reviews (e.g., van Nispen and de Jong, 2017) typically focus on the 
Netherlands as a country with a strongly institutionalized system of policy reviews. The Dutch 
style of policy reviews, or policy appraisals, has also been explicitly labelled as unique among 
European countries (Klein Haarhuis, 2020: 91). 

The origin of policy reviews as procedural tools of ‘involuntary self-assessment’ (van Nispen 
and de Jong, 2017: 154) is to be situated in the framework of attempts to improve the quality and 
pace of financial reporting (Jonker et al., 2016) and performance-based budgeting. Performance 
budgeting in the Netherlands underwent a major reorientation in 2010 with the Accountable 
Budgeting Reform that shifted away from the idealized image of integrated comprehensive con- 
trol of the budget via performance information to a more differentiated approach that focused 
on the benefits and use of information on the effectiveness and efficiency of policies. Also, in this 
reform, there was a shift and thus greater emphasis on the responsibility of ministries for policy 
results. Overall, the picture is one ofa realistic adjustment of expectations regarding the behavior 
of actors involved and a material strengthening of evidence-based decision-making based on 
both performance information and evaluations (Kaiser et al., 2021). 

As it currently stands, the budget serves as the primary structuring mechanism for the orga- 
nization of policy reviews, which tend to be at the level of a budget item. The latter typically 
concerns several programs, subsidies or government agencies. The synthetic nature of the review 
should enable the assessment as to whether the ‘final performance is greater than the sum of its 
parts’ (Jonker et al., 2016). 

Unlike budget reviews, the ownership of policy reviews is in the hands of line ministries and 
so can be applied in a more tailor-made fashion, considering the characteristics (and sensitivi- 
ties) of a particular policy field. The Ministry of Finance stipulates the guidelines (i.e., regula- 
tion periodic evaluations [regeling periodiek evaluatieonderzoek]) and is in charge of the central 
formulation of the terms of reference of the reviews. The guidelines also imply that reviews are 
organized on a rolling basis, with the idea that the whole budget is reviewed every seven years at 
least (van Nispen and de Jong, 2017). The accent is primarily on accountability and legitimacy, 
although learning and enlightenment have more recently been added as additional aims (Klein 
Haarhuis, 2020). Interestingly, both the research design of the reviews and the actual findings 
are submitted to Parliament in the Netherlands. MPs are given the chance to add any questions 
and to comment on the topic and substance before the start of the review. In 2014, Parliament 
requested more insight into potential saving options for inefficient policies which they could use 
in their election manifestos (Jonker et al., 2016). Since then, the requirement has been added 
to identify measures in case of 20% budget cuts. By building in parliamentary reporting and 
feedback mechanisms, opportunities for agenda-setting maneuvering have thus been institution- 
alized. Similar to budget reviews, policy reviews, by design, clearly put the burden of proof on 
those willing to continue current policies. 


129 


Valerie Pattyn and Pirmin Bundi 


Sunset Reviews 


A sunset review is an evaluation of the need for the continued existence of a program that is 

often legally based in an evaluation clause (Balthasar, 2009). A review typically yields a recom- 

mendation to either retain the program, to modify it or to allow the program to terminate, 

including the repeal of relevant statutes (Joint Legislative Audit & Review Committee, 2018). 
Veit and Jantz (2011: 170) define sunset legislation as follows: 


When we speak of sunset legislation, we are not referring to the widespread, case-by- 
case evaluation and limitation of laws, [it] represents an attempt to systematically inte- 
grate the evaluation and limitation of legal norms into the legislative process — either 
in the form of a general obligation to limit and/or evaluate time limits or in the form 
of institutionalized tests of time limits as part of the legislative process. 


Its original can be found in sunset legislation (i.e., temporary laws subject to review) (Fltickiger 
and Popelier, 2022). 

Sunset laws have their origins in deregulation and anti-bureaucratic policies in the United 
States (Ranchordas, 2014: 21 f). Such laws are usually introduced to provide an alternative 
to the increasing volume of legislation, simply by limiting their duration and by inserting an 
evaluation clause conditioning their continuation. For instance, the USA PATRIOT Act (2001) 
was designed as a sunset law that allowed federal prosecutorial powers in the aftermath of the 
9/11 attacks. Underhill et al. (2020) show that sunset clauses often induce a broader range of 
compromise beliefs and significantly more support for conservative legislation compared to 
liberal legislation. In the same vein, Fliickiger and Popelier (2022) argue that the hopes of de- 
bureaucratization were quickly disappointed since sunset laws were often renewed automatically 
without the results of evaluations. 

As Berry (1986) points out, a sunset review consists of the basic elements of an evaluation, 
which assesses both the process and outcome of the legislation. Such reviews allow decision- 
makers to review the usefulness of the current legislation and to make necessary adjustments. 
However, sunset legislations usually do not provide concrete guidance as to the design and 
methods that are required for the evaluation, which is why government agencies often conduct 
evaluations that do not address the need of the specific sunset legislation. Either they focus too 
much on the implementation process (and ignore the outcome and impact of a policy), or they 
provide their report too late so that decision-makers are forced to take action without knowing 
the final result of the review. Beadnell et al. (2017) highlight that soft skills like facilitation are 
equally essential to making sunset reviews work since these enable the production of rigorous 
evaluations that work within the timing of sunset legislation. 

There is an increasing discussion in various contemporary democracies whether and to what 
extent sunset provisions can be used to support the goals of better regulation (Veit and Jantz, 
2011). On the one hand, they provide a legitimation base for government organizations in 
order to conduct evaluations. In Switzerland, for instance, Article 24 of the Energy Utilization 
Decision of October 14, 1990, established the legal mandate to evaluate the action program, 
which was designed as ‘sunset legislation’ (Sager et al., 2017). On the other hand, Veit and Jantz 
(2011) argue that the temporary nature of laws is to be conceived as an attempt to bring about 
a renewed ‘artificial’ agenda-setting at certain points in the political process. It is assumed that 
windows of opportunity for policy change do not open automatically when laws are ineffective 
or generate undesired outcomes. 

By setting a time limit, policymakers have to decide about the continuation of a policy 
after a certain period of validity of a legal norm. This decision can be made on the basis of an 
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evaluation, which triggers evidence-informed policy making (Head, 2016). To sum up, sunset 
reviews provide a tool to structurally not forget policy issues, which can be used strategically by 
actors such as parliaments and government agencies. If we conceive the political agenda as the 
‘list of issues to which political actors pay attention’ (Walgrave et al., 2008: 815), then a sunset 
review functions as a news ticker that pop ups periodically on the political agenda. 


Agenda-Setting Value in Practice? Drivers and Barriers 


This chapter has mainly focused on the value of evaluation tools for agenda-setting without 
considering whether this potential is also met. While assessing their real impact is not straight- 
forward, and differences across polities and tools exist, it is safe to argue that their added value 
should generally not be overestimated. For spending reviews, the limited effect is often built into 
their very design. Even in countries with a long-standing tradition such as the UK or Canada, 
and where a close link between spending reviews and the budget cycle exists, savings usually do 
not exceed 15% of the in-scope spending (van Nispen and de Jong, 2017: 159). 

Other than this, studies have pointed at major problems in implementing reviews, which 
jeopardize their effect. In an OECD (2019a) survey, governments themselves reported a wide 
range of challenges they are confronted with in the implementation of budget reviews, and 
spending reviews in particular. 

The top-reported challenges concern the availability of performance information and the 
poor quality of performance information and data. It is evident that budget reviews very much 
depend on the availability of valid and reliable information, which should enable making 
informed decisions about the effectiveness and efficiency of expenditure. This is not possible 
without a continuous development of performance indicators that can help identifying savings 
options or evaluations that provide evidence on the outcome and efficiency of policy measures. 

Budget reviews are a very resource-intensive activity, requiring that all aspects of the process 
are designed to deliver the best possible return. Governments also indicated a struggle with 
inattention to implementation, time constraints for the implementation, a lack of political sup- 
port and a lack of capacity to organize such budgetary reviews, amongst other, less frequently 
named impediments (OECD, 2019a: 115—116). To maximize use, reviews should be integrated 
as a full part of the budget preparation process, rather than on an ad hoc basis. Additionally, the 
successful implementation of reviews and evaluation requires political leadership at the top level 
that can approve the procedure, set the criteria for the identification of savings and the decision 
on which saving options to adopt (OECD, 2019b). This is not to say, of course, that politicians 
themselves should be involved in the actual implementation of the reviews. On the contrary, the 
fact that review topics can be chosen irrespective of current policies or views of policymakers 
and that ‘out of the box’ options can be named have been mentioned as important conditions to 
make spending reviews succeed (van den Heuvel, 2019). 

For policy reviews in particular, a recent meta-analysis in the Dutch context (SEO, 2018) 
revealed that in only 2% of the review studies could a causal relationship between input and 
outcome be made plausible; in most of the instances (40%), the relationship was qualified as 
‘reasonably accounted for’. Challenges not only relate to methodological difficulties in isolating 
policy effects from external influences on goal indicators and the lack of effectiveness evalua- 
tion in individual studies underlying policy reviews but can also be attributed to a mere lack of 
evaluation knowledge and capacity among commissioners of the reviews. Additionally, existing 
policy reviews often lack information on alternative policy options (efficiency). Interestingly, the 
study also pointed at strong disincentives for line ministries to share information on ineffective- 
ness, which is more a cultural hurdle to overcome (van den Heuvel, 2019). Knowing that the 
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Netherlands is among the highest-scoring European countries on indices of evaluation maturity 
(Furubo and Sandahl, 2002; Stockmann et al., 2020), the challenge is probably even more pro- 
nounced for countries without such tradition. 

As to sunset reviews, the four-country study of Veit and Jantz (2011) in the US, Australia, 
Germany and Switzerland is insightful. Even though the US is considered a pioneer in sunset 
legislation, sunset reviews were abolished in several US states after only a few years. The main 
reason was the immense amount of resources required by legislatures that often only work 
part time (Kearney, 1990; Squire, 2017). Due to work overload, laws were often automatically 
extended after the end of the term without the legislature exercising any actual oversight func- 
tion. In addition, the use of fixed-term clauses did not lead to substantial closure of government 
agencies, nor did it save any significant amount of budgetary resources. 

Hence, general fixed-term clauses proved to be ineffective and unsuited for the federal states, 
which were the main target of sunset legislation in the United States. In contrast, Australia 
knows a general time limit requirement for sub-statutory regulations at the central government 
level (Legislative Instruments Act). Laws are valid for ten years before they are evaluated by the 
authorities. However, an internal evaluation showed that the actual evaluation activities of the 
government organizations have so far been inadequate in many cases, despite comprehensive 
evaluation obligations and support materials. 

A pragmatic approach to evaluation requirements, varying from case to case, seems to prevail 
with recent attempts, particularly at the sub-state level, to establish the level of administrative 
burdens as a decisive criterion for the intensity of evaluation activities (McMillan and Ombuds- 
man, 2009). Also in Germany, sunset legislation is widespread across federal states. Administra- 
tive regulations are even subject to a general time limit requirement in ten federal states, which 
is typically five years (Jantz and Veit, 2010). Even though there is an obligation to check all draft 
laws of the federal government ex ante as to whether time limits are possible and whether an 
evaluation should be carried out, Veit (2010) shows that this has been implemented inadequately 
in practice. In Switzerland, finally, the obligation to evaluate public measures is not linked to a 
general obligation to set a time limit. However, the Federal Office of Justice explicitly includes 
the possibility of setting time limits for laws and formulates a corresponding list of criteria 
(Bundesamt fiir Justiz, 2019). Accordingly, laws should be limited in time if they deal with 
temporarily occurring problems or problems that can be permanently solved by other suitable 
measures after a certain period of time. 

Furthermore, time limits are advisable for laws with uncertain effects, for laws with high 
financial costs and for laws that are to be periodically reviewed with a view to systematic impact 
control. Wirths et al. (2017) show that more than three-quarters of evaluation clauses are already 
introduced by the administration in the bill. In most cases, evaluation clauses are then seen as a 
promising tool for evidence-based governance of public services and policy programs. 

Of course, a successful implementation of reviews and evaluations does not guarantee use 
and an actual impact on governments’ agendas. Research has shown that there is much waste 
of reviews and evaluations: results from high-quality evaluations are not always used, even in 
settings with high evaluation maturity (Pattyn and Bouterse, 2020). For instance, in a study on 
the use of evaluations and policy reviews produced between 2013 and 2016 by the Policy and 
Operations Evaluation Department (IOB) of the Dutch Ministry of Foreign Affairs (i.e., a typi- 
cal case of relatively high evaluation maturity), only five out of eighteen studies produced were 
perceived to be used, at least in an instrumental way (Pattyn and Bouterse, 2020). 

Such observations confirm that policymaking is highly complex and that it would be naive 
to think that evaluations and reviews would be used in a linear and rational way. Of the plethora 
of conditions that may influence use, this study highlighted the importance of interest among 
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policymakers for the evaluation or review and engaging them in the evaluation process. It is not 
surprising, of course, that an evaluation will only be used when the main policymaker(s) are 
really dedicated to doing so. By the same token, it was shown that timing matters a lot and that 
reviews and evaluations have more potential to make a difference when taking place simultane- 
ously with drafting new/changed policy measures (Pattyn and Bouterse, 2020). 

While some evaluations and reviews have been shown to result in tangible changes of govern- 
ments’ agenda, and whereas evidence exists of reviews having impact on election programmes of 
political parties (see van den Heuvel, 2019, for examples of the Dutch case), results from evalu- 
ations or reviews are often merely ‘used as additional background information, or as political 
ammunition in budget negotiations both for extra spending or spending cuts, and [are] simply 
ignored if not useful’ (van Nispen and de Jong, 2017: 159). 

One cannot ignore the very politics of evaluations and reviews, which is unavoidable given 
their close entangling with the policy process. As a matter of fact, and given the increasing 
institutionalization of such procedural tools, both political and administrative actors over time 
learned to make the best of evaluations and reviews by using them not only as reflective tools with 
epistemic relevance to improve policy making, but also as political instruments helping them 
pursuing their own agendas (Mavrot and Pattyn, 2022). Evaluations and reviews by nature lend 
the rational arguments to strategically package and justify predefined administrative or political 
priorities in a seemingly value-neutral, ‘evidence-based’ way. Of course, strategic ‘underuse’ or 
‘misuse’ of reviews and evaluations is not an argument to throw the baby out with the bathwater. 


Conclusion 


The family of tools discussed in this chapter share the characteristic that they permit issues (back) on 
the agenda at set intervals (Bali and Halpin, 2021). Given their procedural anchorage, each of the 
tools de facto requires those wishing to continue the policy to justify this position. As can be derived 
from the summary overview (Table 10.1), the instruments vary in the extent to which they also put 
other policy ideas on the agenda or whether they are rather backward looking and reactive only. 
Each of the tools addresses different dimensions and hence provides other information to 
decision-makers. For instance, budget reviews are very much focused on the potential savings 
for the public budgetary and therefore rather focus on efficiency. Policy reviews are mostly 


Table 10.1 Reviews as Policy Tools for Agenda-Setting 


Principal Budget Reviews Policy Reviews Sunset Reviews 
Cabinet Committee Spending Department Legislator 
Aim Potential savings (spending Arguments/ Accountability/ 
reviews) and proposals for extra information information 
funding (expenditure reviews) 
Focus Spending Policy relevance Goal attainment, 
effectiveness 
Unit of Analysis Policy Program Program/organization 


Impact on Agenda Reactive and proactive, leaving 
some room for new policy 


ideas 


Mainly reactive, 
leaving little room 
for new policy ideas 


Source: Authors’ elaboration, based on van Nispen and de Jong (2017: 155) 
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concerned about arguments and information and discuss the relevance of policies which leave 
organizations more methodical leverage than the other reviews. Finally, sunset reviews are aimed 
at addressing the effectiveness of public measures and whether they have reached their foreseen 
objectives. 

Over time, the use of such procedural evaluation tools has been strongly institutionalized in 
the public sector, to the extent that scholars increasingly talk — often in a pejorative way — about 
emerging ‘evaluation systems’ (Leeuw and Furubo, 2008) in which the application of such 
tools has become strongly routinized practice. It is particularly this often uncritical routinization 
which made observers question their added value in many instances (Raimondo and Leeuw, 
2021). Our analysis shows that, independently of the political system, the implementation and 
use of reviews and evaluations are indeed very ad hoc. They have definitely not reached the 
point where they set the agenda systematically and regularly. Amongst other elements, organi- 
zational capacity proves essential for successful implementation. 

Irrespective of mere implementation challenges, the critical question also arises as to what 
the optimal volume of review is that governments can engage in. With decision-makers often 
having many studies at their disposal already, they risk lacking the time to process all relevant 
information, even if such information is of high quality. Moreover, timing is critical. Reviews 
can only play a comprehensive role in agenda-setting if they are already taken into account dur- 
ing the legislation process. 

In the future, governments may take advantage of the use of modern ICT methods to sys- 
tematically use review information (Cahlikova and Ballester, 2022), provided that the contextual 
contingencies and autonomous learning ‘problems’ of AI are overcome. But again, with evalu- 
ations and reviews being intrinsic political undertakings, it will ultimately be a strategic — not 
merely a technical — decision whether governments are willing to walk this path. 
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There are different theoretical perspectives on why policy actors promote certain policy propos- 
als, but they all have in common that a policy idea needs support — from the public, key actors 
in government and administration, or influential corporatist actors such as interest groups — to 
be successful. How do policy actors gain supporters for their ideas in the agenda-setting phase? 
Which tools do they use to promote their ideas at this stage? This contribution argues that 
policy actors can make use of the career motivations and social psychological drivers of actors’ 
behavior to achieve popular mobilization and social advocacy for their ideas. In detail, we set 
out the mechanisms that link a policy idea successfully entering the agenda-setting stage to the 
tools used by policy actors by drawing on the lens of the programmatic action framework (PAF). 
We thereby distinguish four types of tools through which actors seize the attention of decisive 
actors and promote their policy proposals: those that target the salience of social identities of 
policy actors that favor policy proposals (social identity-driven tools), those that target the nor- 
mative implications of policy proposals (program-driven tools), those that achieve issue attention 
through the exertion of authority and motivate actors in promising top-level positions (career- 
driven tools), and those that build on the creation of knowledge and evidence in the policy 
process (education-driven tools). We also discuss the delineation and overlapping of these tools 
with regard to existing classifications of policy tools and provide practical advice to use these 
tools for ensuring long-term support of a policy program by a diversity of actors throughout 
the policy cycle. 


Introduction: The Need for Actor-Centered Policy 
Tools in Agenda-Setting 


Policy instruments and policy tools are consistently defined as the means and techniques 
deployed by policy actors to achieve policy goals (Howlett 1991; Capano and Howlett 2020). 
Depending on the stage of the policy process a policy issue finds itself in, however, goal achieve- 
ment may refer to different things. In the dominant and widely used understanding of policy 
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instruments as concerned with effective policy implementation, policy goals describe the desire 
for certain outcomes. In the agenda-setting phase of the policy cycle, though, the goals of policy 
actors rather lie in raising public awareness of a policy problem among both policy and societal 
actors and ideally already gathering support for a preferred policy instrument (or tool mix) in 
combination with a presented policy idea (Zahariadis 2016). 

There are different theoretical perspectives on why policy actors promote certain policy 
proposals, but they all have in common that a policy idea needs support — from the public, 
key actors in government and administration, or influential corporatist actors such as interest 
groups — to be successful. How do policy actors gain supporters for their ideas in the agenda- 
setting phase? Which tools do they use to promote their ideas at this stage? Despite a range 
of existing classifications to delineate different types of policy instruments, there is so far only 
limited attention to procedural policy instruments that have an impact on policy-making pro- 
cesses and policy actors’ behavior (Bali et al. 2021). Focusing explicitly on those discussed in the 
context of agenda-setting, we add an actor-centered perspective to the tools used by actors to 
mobilize the public and foster social advocacy for policy proposals. 

Due to its actor-centered perspective, the classification that is developed in this contribution 
takes into account psychological insights on the preference formation and behavior of actors. 
The integration of (social) psychological theories into public policy research has a long-standing 
tradition, with its beginning prominently marked by the work of Herbert Simon (1976) on 
(procedural) rationality as rational individual behavior bounded by institutional constraints. 
Since then, the concepts of belief systems, personality traits, and social groups have been sub- 
sequently integrated in part in theoretical explanations of policy change (Weible and Sabatier 
2009; Zafonte and Sabatier 1998; Freitag and Bauer 2016; Schwartz 2021; Bandelow and Hor- 
nung 2021a). As a consequence, this contribution argues that policy actors can make use of the 
cognitive drivers of preferences and behavior, such as group identification, norms, the desire for 
authority, and rationality, to achieve popular mobilization and social advocacy for their ideas. 

In detail, we set out the mechanisms that link a policy idea successfully entering the agenda- 
setting stage to the tools used by policy actors to gather support for this idea. Drawing on the 
lens of the programmatic action framework (PAF) (Hassenteufel and Genieys 2021; Bandelow 
et al. 2021), the tools set out in the following refer to a long-term perspective on policy ideas 
following a cyclical pattern of programmatic dominance. They are thus oriented not just towards 
tool for raising short-term attention, but to bringing actors following a policy proposal from 
agenda-setting to eventual implementation. 

In doing so, the contribution first describes the cyclical understanding of policy ideas similar 
to “economic cycles” (Bandelow and Hornung 2021b). Subsequently, a classification of agenda- 
setting tools integrating (social) psychological insights in a long-term view distinguishes four 
types of tools through which actors seize the attention of decisive actors and promote their policy 
proposals: those that target the salience of social identities of policy actors that favor policy pro- 
posals (social identity-driven tools), those that target the normative implications of policy propos- 
als (program-driven tools), those that achieve issue attention through the exertion of authority 
and motivate actors in promising top-level positions (career-driven tools), and those that build on 
the creation of knowledge and evidence in the policy process (education-driven tools). 

The classification is delineated from existing typologies of policy tools. Besides the theoreti- 
cal considerations, it also provides practical advice to use these tools for ensuring long-term sup- 
port of a policy program by a diversity of actors throughout the policy cycle. This also includes 
the question of who should be the actors that mobilize the public and recruit supporters. We 
conclude by formulating a research agenda for future research on policy tools to be used from 
the agenda-setting stage onward. 


138 


Popular Mobilization and Social Advocacy 


A Cyclical Understanding of Policy Ideas 


What is important to observe is the temporal dimension of a policy idea. It may be that a policy 
issue is only shortly on the agenda and almost instantly disappears again, as was the case during 
the COVID-19 pandemic, which opened opportunities for change left unused by policy actors. 
Per Bodenheimer and Leidenberger (2020), windows of opportunity have to be accompanied 
by long-term strategies to sustainably promote change in a policy area. 

While it is certainly as relevant to investigate the conditions under which policy ideas fade 
away after having only reached the stage of agenda-setting, there is a long-term, strategic ele- 
ment inherent in policy design that poses the question of what tools actors use to promote 
popular mobilization and social advocacy in the long term. Only then is the agenda-setting 
phase the beginning of an ongoing process of policy formulation, design and redesign, and final 
implementation. Also, if general attention to a policy issue is maintained, this may be conducive 
to initiating a debate around the adequate design and instrument mixes addressing the identified 
policy problem. 

The perspective of the PAF sets out a cyclical understanding of policy issues and ideas. It 
argues that these are connected to the careers of policy actors, and therefore policy actors have 
an interest in not just bringing their policy idea onto the political agenda but also in promot- 
ing it throughout the policy cycle to eventually reach career positions with this program. The 
notion of cyclical processes of idea promotion is not new but anchored in the issue attention 
cycle. However, the issue attention cycle, an observation made by Downs (1991), has less focus 
on actors than on issues and argues that an issue runs through different stages of public atten- 
tion, which are largely driven by the nature of the issues and the media logic. Yet how policies 
rise and decline, from a PAF perspective at least, is a matter of actors surrounding these ideas as 
well. From an actor-centered perspective, it is therefore interesting to study why actors stick to a 
certain policy idea, and which tools can be used to foster their support from the agenda-setting 
stage onward. 


Four Drivers of Advocacy 


The question why actors support certain policy proposals and not others has been subject to 
an ongoing research endeavor in the policy sciences. It is connected to the understanding and 
explanation of the underlying functioning of human cognition as well as human preference 
formation and subsequent behavior. Being able to understand why actors evaluate a problem as 
pressing, putting it at the top of their agenda to solve it, or evaluate an instrument as particularly 
effective, striving toward putting it on the agenda of everyone else, is still puzzling. 

An often-studied element of agenda-setting dynamics is that of a sudden shift in attention. 
This may be due to disasters (Crow et al. 2018), a focusing event (Birkland 2004), or a window 
of opportunity (Herweg et al. 2015), which stresses the hazardous consequences of inaction. 
Such focusing events have been found to also effect the degree to which less powerful groups 
mobilize and may actively and successfully advocate change in public (Birkland 1998). Recent 
research questions the necessary suddenness of such events, arguing that it might also be aggre- 
gate focusing events that lead to an increase in attention over time (O’Donovan 2017). Beyond 
the idea of exogenous happenings that affect the urgency with which individuals want to see 
a problem solved, however, there is also a need to turn to the endogenous drivers of agenda- 
setting and the tools that actors may actively employ rather than waiting for an event to come. 
This also adds to the literature on low-salience policies and the question of how these can be put 
on the agenda if they have not yet reached public opinion (Koski 2010). 
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Previous research on agenda-setting tools is rather scarce. A prominent existing classification 
stems from Hood (1986) and identifies nodality, authority, treasure, and organization as potential 
tools to mobilize the public. Nodality includes information tools such as government com- 
munication; authority includes any action taken by government to force anew attention to an 
issue, such as periodic reviews and sunset clauses; treasure relates to any financial means that the 
government uses to incentivize a specific behavior that sets the agenda for an issue; organization 
refers to the way in which the government uses its organized procedures to set an agenda: for 
example, by anticipating future developments through scenario building (Howlett and Shivakoti 
2014). 

The typology of Hood is frequently applied in current studies of agenda-setting tools (Faw- 
cett 2021). Bali and Halpin (2021) use it as an orientation for their identification of agenda-set- 
ting tools to shape engagement from organized interests to manage policy demands. Other types 
of agenda-setting tools are questioned to fall under the nature of being procedural or substantive, 
as Stark and Yates (2021) argue for public inquiries, which are beyond the government’s control. 
Other tools, such as open collaboration platforms for crowdsourcing, fit well in a range of the 
categories listed by Hood (Taeihagh 2017). 

While existing typologies present important and useful steps in distinguishing (procedural) 
agenda-setting tools, it is so far understudied in what way actors actively make use of the cogni- 
tive mechanisms in individuals’ minds, thereby using tools to set the agenda and to foster the 
mobilization for and advocacy of their preferred policies. It is this theoretical gap that the fol- 
lowing classification contributes to, by theoretically deriving four types of tools and connecting 
them to the mechanisms through which actors may achieve long-term support for policy ideas 
that lasts beyond the sudden shifts in attention usually observed at the agenda-setting stage. 


Social Identity—Driven Tools 


Only recently has the perspective of social groups and social identities found entry into policy 
studies (Hornung et al. 2019; Hogg et al. 2017). With a view on the dynamics of the policy 
process and the explanation of policy change, the focus on individual social identification has 
shed light on paradigmatic change (Hornung and Bandelow 2020) and voting behavior (Vogeler 
et al. 2020). 

The main idea is that actors orientate their preferences and behavior according to their salient 
social identity, meaning that depending on which group membership they are most aware of in a 
given moment, they will think and act differently (Turner 2010; Lalonde and Silverman 1994). 
Hornung et al. (2019) distinguish five general types of social identities that may be relevant in 
policymaking: organizational identities (such as political parties or associations), sectoral iden- 
tities (such as committees or sector-specific networks), local identities (such as attachment to 
the hometown, state, or city), demographic and biographic identities (such as age, profession, 
or ancestry), and informal identities that form on the basis of informal connections and often 
emerge from the previous group memberships. 

The group-theoretical perspective can be used in the agenda-setting phase because it allows 
policymakers to target individuals by triggering their group salience. Past and present research 
in social psychology has centered on the question of what triggers the salience of social identi- 
ties, with answers including the language used (Morris et al. 2008), the targeting of campaigns 
(Dickson and Scheve 2006), social context (Ellemers et al. 2002), and events (Chrobot-Mason 
et al. 2009). Explicitly transferred to public policy research, Mols et al. (2015) add to the lit- 
erature on modes of governance to outline that a long-term change in an individual’s behavior 
and preferences requires altering the internalization of norms. This is best achieved, they state, 
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after depicting social relations and salient identities and understanding the norms and where they 
come from, to afterward address these identities in the agenda-setting process. 

In detail, this means that when setting the agenda, policymakers may make use of different 
existing groups and their social identities to directly target individuals via their group member- 
ship. Framing individuals as members of a specific group and outlining the importance that a 
policy issue has for that group are essential factors for how they will evaluate ideas (Pepermans 
and Rousseau 2021). This comes also close to the notion set forward by Birkland (2007) that 
agenda-setting is a result of a struggle between groups advocating their norms to shape solutions 
and policies in the long term. 

A special case of a social group exerting influence on the policy process is that of a program- 
matic group. A programmatic group is labelled according to the program it attaches to and with 
which it is irrevocably tied (Bandelow et al. 2021). Programmatic groups are often the result of 
biographical intersections of individuals: e.g., working commissions or committees. Therefore, 
besides using existing social group memberships and the identities they shape to trigger and 
transform certain ways of thinking on the side of individuals, policy actors may also actively 
create groups by establishing them through councils or commissions, which then also set the 
agenda for a certain topic. 


Program-Driven Tools 


Actors may also advocate a policy idea or instrument because they normatively think it is the 
best to choose. Some scholars might argue that individuals possess firm beliefs that have been 
built from socialization (Searing et al. 1973), but recent research in psychology questions this sta- 
bility of norms and values. Instead, it supposes that actors may in general support any norm, and 
even opposite ones within shorter periods of time, just because the peers they regularly engage 
with change (Parsons 2015). Nevertheless, individuals may think that their norms are stable 
and that attaching to a certain policy idea is morally right and superior, to which psychological 
research on “good and evil” (Miller 2016) provides more insights. 

The programmatic action framework (PAF) hypothesizes that the normative attachment to 
policy ideas up to cohesive programs stems not from normative convictions but from the identi- 
fication with other actors surrounding the policies. This is similar to what the policy instrument 
literature labels instrument constituencies (Béland et al. 2018; Simons and Voß 2018), with the 
important difference that the source of attachment is not the instrument but the identity connected 
to it. As a consequence, actors do not cherish the instrument itself but the feeling of belonging 
to a group that dominates the debate in the search for policy alternatives and the resulting power 
over others, combined with the feeling of (often moral) superiority. In general, this is good news 
for policy actors striving to gather support for setting their agenda because it implies that individu- 
als are generally ambiguous toward preferences and can be won over for any policy by providing 
them with a social group that they can identify with and that suggests the support of this proposal. 

How can policy actors use this normative, program-driven tool to achieve popular mobi- 
lization and social advocacy for their ideas? First of all, the communication of a problem or a 
policy plays a key role. Making people feel that a problem is severe and that solving it must be 
put on the top of the political agenda can spark an ongoing interest that outlives until the final 
implementation stage. Thereby, it is necessary to convey that anyone who equally notices this 
problem belongs to those who truly recognize the importance of a policy issue. An example is 
presented by the historical corporate tax avoidance of companies operating in Australia, where 
campaigners have successfully framed a reform of this “deficiency” as adhering to the principles 
of fairness, morality, and global equality (Murphy-Gregory et al. 2020). Tosun and Scherer 
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(2020) stress the importance of a dominant policy image for setting the agenda. As opposed to 
earlier studies, the influence of the media on setting the agenda is meanwhile seen from a dif- 
ferentiated perspective (Dalton et al. 1998). Even if morality arguments are put forward, as was 
the case in the Clinton/Lewinsky scandal, these have to be connected to a policy issue and an 
electoral or government agenda to be impactful (Yioutas and Segvic 2003). 

The second program-driven tool is continuously mentioning an issue in public and in con- 
versations with other policy actors, bureaucrats, and sectoral actors. Psychological research 
informs us that often-heard concepts are remembered and internalized in the long run, thereby 
slightly increasing support (Ernst et al. 2017; Cacioppo and Petty 1989). At least in the area of 
climate policy, Muradova et al. (2020) find that proposals have a higher chance of gaining public 
support if they are repeated by multiple speakers in the Irish Citizen’s Assembly. 

At first sight, one might think that program-driven tools are only suitable to attract a specific 
target group of bureaucrats and the general public because both policy actors and sectoral actors 
have either economic or other ideologically determined interests that prevent them from pro- 
moting an issue no matter the frame if it counters the views of their primary affiliation. Yet, the 
“Nixon-goes-to-China” effect teaches us differently, outlining that actors may well take stances 
that usually lie on the opposite site of their alleged preferences. It is therefore in general always 
possible to also normatively convince people that a policy issue is important or a policy idea is 
worth pursuing if conveying the impression that it is guided by information (Sulitzeanu-Kenan 
and Zohlnhofer 2019). 


Career-Driven Tools 


An inherent assumption in the study of programmatic action is that policy actors — above all, 
civil servants and bureaucrats, but also sectoral actors and party politicians — strive towards 
increased authority in their policy sector. This assumption is widely backed by public adminis- 
tration research, showing that those graduates who pursue a career in government are motivated 
by a wish for impact (Chetkovich 2003) and thus driven by (also intellectual) self-interest rather 
than ideology (Egeberg 1995; van der Wal 2013). A third tool to be used in the agenda-setting 
phase can thus be seen in achieving issue attention by motivating actors through promising them 
top-level positions. Following up from the previous two presented types of tools, this is strongly 
connected to the idea of the economic cycle of policy ideas and the idea that careers depend on 
programs and public commitment to such programs. 

While the perspective of the PAF hypothesizes actors’ preferences and behavior as mainly 
driven by social group memberships, particularly informal identities, critical voices might 
raise the question why, from a group-theoretical view, the individual motivation plays a role 
in agenda-setting. By referring to the self-categorization theory (SCT), the PAF argues that 
structuring human beings into social groups and categorizing oneself with reference to social 
groups to fulfill both the purpose of distinction from others and the search for sameness (optimal 
distinctiveness theory) (Brewer 1991) are integral parts of human behavior and thinking. This 
does not mean that individuals do not have motivations that lead them to actively join social 
groups or that they do not use social relations to fulfill their own goals. As a consequence, using 
the self-interest of actors may foster support for a policy issue or idea because they see this as 
an opportunity to use it for their own career advancement. This is particularly the case as the 
solution to problems that enter the agenda is sometimes achieved through outsourcing and 
sometimes through the establishment of job positions, departments, or agencies. When these are 
institutionalized, they also offer long-term career trajectories for those actors who are attached 
to it (Duque 2021). 
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What does this mean in detail? Policy actors, constantly striving toward career enhancement, 
need to at the same time stick out from the masses but also not polarize. This is also the reason 
agenda-setting is best carried out by a group of people in which responsibility is diffused rather 
than concentrated in only one policy entrepreneur whose prominence rises and falls with the 
policy idea. A group of actors in diverse positions has greater potential to introduce an issue at 
diverse points of the political system and to use it collectively for career advancement across the 
different institutions. To trigger the career aspirations of individuals, policymakers can (1) for- 
mulate job advertisements that correspond to their preferred policy (such as “climate mitigation 
observer”), (2) provide further education for employees to learn more about the topic that is to 
be put on the agenda, or (3) introduce new study programs into universities that educate people 
accordingly so that they will promote certain policies throughout their own career paths. 


Education-Driven Tools 


Connected to the education of individuals is the idea that policy tools used to promote popu- 
lar mobilization and social advocacy can build on the integration of scientific insights served 
to foster knowledge and use arguments of evidence in the agenda-setting stage. When trying 
to convince individuals why a certain problem is important or a certain solution is the best, 
including scientific support in the communication helps justify and legitimize the agenda (Ettelt 
and Hawkins 2018). With a view on programmatic groups, this again is connected to the idea 
of social identities and biographies, rather than on the ideas of the “myth” of evidence-based 
policy-making (Cairney 2020). 

Regarding biographies, it comes as no surprise that individual actors are socialized by their 
education and that the things they learned at university and the subjects they studied influence 
the way in which they think about the world in general and policies in particular. For example, 
students of law will have different answers to the question of how to organize health care than 
health economists or doctors. These views are not be confused with firm beliefs that developed 
during the socialization phase but constitute a way of how to see the world stemming from 
regular interaction with peers studying the same subjects and/or learning the same arguments. 
Therefore, the views are much less stable and, in general, may change quickly when the indi- 
viduals are confronted with different scientific arguments and actors with different backgrounds 
with whom they have to work in a policy sector. 

Turning to the notion of social groups, one way of including evidence in the policy process 
is the institutionalization of scientific advice and the establishment and use of advisory systems 
(Laage-Thomsen 2021). These may also serve the function of social groups and shape an iden- 
tity when there are diverse actors bringing in their expertise and developing a governmental 
strategy. In that, their impact follows from content (policy programs or setting policy issues on 
the agenda) rather than from the location where the scientific advice is bound in (Craft and 
Howlett 2012). The institutionalization of expert advice has been found to be a main driver of 
the development of policy programs and, relatedly, to the formation of programmatic groups 
(Hornung 2021). From a policy tool perspective and related to the typology set out by Hood 
(1986), the regular inclusion of scientific advice in agenda-setting processes corresponds to the 
authoritative category of policy tools. 


Agenda-Setting Policy Tools in Context 


Summarizing the typology laid out in the previous sections, four general types of agenda-setting 
tools that are procedural by design can be imagined. Figure 11.1 visualizes this typology. The 
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Figure 11.1 PAF Typology of Agenda-Setting Policy Tools 


Source: Authors’ visualization 
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step-wise list of tools implies and suggests that the four types of tools are ideally used in com- 
bination with each other, within a coherent strategy of agenda-setting. However, this does not 
exclude the possibility that single tools can be used separately from the others, depending also 
on the context one finds oneself in. 

Regarding context, the last line of the table includes suggestions for the situations in which 
some tool might be more suitable than another. The social identity—driven tool functions via 
the mechanism of making individuals feel that they belong to their own group or triggering 
the salience of their already-existing identities. It is suitable for the policy context in general 
because it can be assumed that policy actors possess different social identification, of which it 
is relatively easy to find one to trigger. Program -driven tools are particularly suitable in con- 
texts in which there is a morally superior or an emotional component involved, such as animal 
welfare policy. In such cases, program-driven tools may well be used to frame or reframe an 
issue accordingly to put it on the agenda. Career-driven tools are particularly powerful in 
policy sectors that offer financial or authoritative resources for the individuals working in 
the sector because policy actors can make them believe that it will be helpful career-wise for 
them to promote a policy issue. Finally, education-driven agenda-setting tools function over 
the support provided by advisors and scientific expertise; therefore, they are most appropriate 
to be employed in contexts of highly complex subjects. Policy sectors with issues that require 
a substantial degree of scientific expertise will find it easier to use scientific advice actively in 
setting the agenda. 

Besides the context, which agenda-setting tool is used with the best effect is also a ques- 
tion of the target group. Nearly all policy tools can be employed for policy actors and sectoral 
actors. Program-driven tools, however, are more effective in convincing the public and non- 
political actors, as these have much less knowledge and fewer fixed preferences than policymak- 
ers. Career-driven tools, on the other hand, only address people who actually strive toward 
making a career in the policy sector, such as bureaucrats, policy actors, and sectoral actors. Social 
identity—driven tools and education-driven tools can be used for all sorts of clientele because 
both the cognitive mechanisms of group belonging and the processing of newly presented evi- 
dence function the same way. 


Implications for Practice and Where to Go From Here 


While these classifications are clearly a starting point for future research on the tools used by 
actors to bring issues onto the agenda, there are direct implications — at least in a theoretical 
manner — that can be derived from the aforementioned considerations. These essentially con- 
cern a hands-on guide on how to build a programmatic agenda and not just putting an issue on 
the political agenda for a short time, but to let it flow through the entire economic cycle policy 
ideas may go through (or not). It is no coincidence that the tools have been presented in a cer- 
tain order, and they can almost identically be integrated into a step-wise process of gathering 
support among the public and organizing social advocacy. 

The programmatic action framework provides a lens on policy processes that turns attention 
to social group memberships and biographies as drivers of individual preferences and behavior. 
Thereby, the present contribution derives from the assumptions of the PAF policy tools that 
can be used in the agenda-setting phase of the policy cycle to promote popular mobilization 
and social advocacy for policy issues and ideas. While the classification of agenda-setting tools 
remains theoretical for now, it strongly encourages empirical research on the question of which 
types of these agenda-setting tools are used alone or in combination, for which target groups 
they are used, and in the context of which policy sector. 
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CONSULTATION TOOLS 
AND AGENDA-SETTING 


Bert Fraussen 


The recent scholarship in the policy sciences has highlighted how different policy instruments 
and tools are assembled or bundled in complex policy portfolios in different stages of the policy 
process. A nascent strand of this important work concerns the agenda-setting phase, in which 
scholars aim to understand the instruments — procedural and substantive — that the government 
uses to shape the issues that it has to address. This chapter presents a taxonomy to typologize 
agenda-setting instruments and illustrative examples of emerging policy mixes used to man- 
age policy demands. Tools are classified as those which governments use to routinize demands, 
regularize demands, generate demands, and impose issues onto the agenda. The chapter makes 
two contributions to the literature on policy tools and mixes. First, it presents a framework to 
study agenda-setting tools, and second, it contributes to the literature around policy portfolios 
by presenting empirical examples of emerging policy mixes used in the agenda-setting phase of 
the policy process. 


Introduction 


This chapter focuses on the consultation approaches and tools government have at their disposal 
to manage the agenda-setting phase and organize consultation process and, in this way, shape 
the participation and formulation of policy demands by external stakeholders, such as interest 
groups, think tanks, companies, citizens, and experts. While much work has focused on how 
and to what extent these external stakeholders have the ability to set the government agenda, 
the other side of the equation — namely, how policymakers approach stakeholder engagement in 
this early phase of the policy cycle — has received much less attention. 

In all phases of the policy cycle, policymakers are in need of policy expertise and insight into 
societal legitimacy of policy measures, and the agenda-setting phase obviously is no exception. 
Quite in contrary, this kind of information might be most critical and influential in the earliest 
moments of setting the agenda and developing policy proposals as the space to shape the content 
and scope of policies is arguably largest at this point. This implies both that external stakeholders 
see this as a unique window of opportunity to shape policy initiatives, and public officials are 
likely to be most open to suggestions and input, compared to later stages in the policy process. 

In organizing the consultation process, governments face several difficult choices and trade- 
offs, which will be addressed in this chapter. The first question involves which consultation 
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approaches and tools are most suitable, a question that has only become more important as the 
number of available policy tools has proliferated. The second question relates to the objectives 
of the central actors: Which strategies can governments follow to manage stakeholder demands, 
how do bureaucratic motives shape the chosen consultation approach, and how do stakeholders 
approach and perceive different consultation tools? These strategies and choices also relate to key 
trade-offs, such as the fairness and inclusiveness of consultation procedures. 


Consultation Tools and Approaches 


Consultation of external stakeholders is a central feature of contemporary governance. Much 
academic work has focused on the consultation of external stakeholders, such as NGOs, firms, 
business associations, think tanks, and citizens in the policy formulation phase. In this phase, 
consultation procedures are often characterized by higher transparency and more strongly insti- 
tutionalized. Yet the importance of consultation practices at both earlier and later stages, such 
as agenda-setting, implementation, and evaluation, should not be overlooked. The phase of 
agenda-setting in particular provides a key opportunity for public officials and external stake- 
holders to set the policy scene and determine which (aspects of) policy problems receive atten- 
tion and the range of policy solutions put forward for political and public debate. 

The increased importance of consultation procedures to both policymakers and stakeholders 
can be explained by a combination of developments. As highlighted by Bunea (2020: 2), a com- 
bination of government reforms triggered by ideas related to New Public Management (acceler- 
ated by the emergence of new digital technologies), increasing concerns about the legitimacy 
and accountability of public policies, a growing need to enhance the functioning of regulatory 
regimes, more emphasis on evidence-based policymaking, and better regulation has resulted 
in more attention to consultation processes, greater investments in these procedures by public 
officials, and more scholarly attention. 

In recent years, the number and variety of consultation tools that policymakers have at their 
disposal have increased considerably. Classic tools, such as advisory councils, parliamentary hear- 
ings, and expert committees, have been complemented with new instruments. Some of these 
tools have been enabled by new technologies, such as the increased use of online public con- 
sultations. Other tools follow a trend towards more participatory and collaborative modes of 
policymaking, such as cocreation, codesign, and collaborative policy platforms. 

As a result, policymakers have a large toolbox and a variety of tools to engage with external stake- 
holders. To provide one specific example, in recent communication, the European Commission 
(see European Commission 2017) distinguishes the following consultation tools: (1) conferences, 
public hearings, and events; (2) Eurobarometer surveys; (3) Expert groups of the Commission; 
(4) focus groups; (5) interviews; (6) public consultations; (7) consultations targeting SMEs-SME 
panel; and (8) workshops, meetings, and seminars. Excluded from this list are direct meetings with 
policymakers. While this cannot be considered a consultation tool, this form of interaction between 
external stakeholders and policymakers still constitutes a scarce and highly valuable channel as it 
involves privileged access for a select number of groups, often on an individual basis. 

While there is a tendency to study consultation tools in isolation (in particular those linked to 
democratic innovations), the political and administrative reality is that within a single policy pro- 
cess, policymakers often combine several consultation tools to involve stakeholders and acquire 
policy input. As a result, different types of tools are used to engage with external stakeholders 
in a similar issue context. In recent work, Fraussen et al. (2020) consider how public authorities 
might combine these different tools and conceptually distinguish between open, closed, and 
hybrid approaches to engaging stakeholders in policy processes. 
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The use of internet consultations provides a good example of an open consultation approach 
as this tool enables the participation of all external stakeholders who seek to provide policy 
input. An open consultation approach implies that the consultation procedure is accessible to 
everyone. Hence, theoretically, the number of external stakeholders who can participate in an 
open consultation approach is unlimited, as it provides “self-selected” involvement for a wide 
range of possible participants (2020: 476). An open consultation approach seems most suitable 
when policymakers aim to consult a very large and broad set of stakeholders. 

The main difference between an open consultation approach and a closed consultation 
approach relates to the role of (elected or unelected) policymakers as gatekeepers. When a con- 
sultation approach is closed, participation depends on being invited by policymakers to partici- 
pate in the consultation process: for instance, being selected as member of an advisory council, 
participant in a workshop, or expert on a specific committee. While many external stakeholders 
may wish to participate, places are limited, and only those stakeholders who are considered 
“relevant” participants by policymakers will be granted a seat at the table. Inclusion in these poli- 
cymaking venues is often considered a privileged form of access as the smaller set of participants 
combined with the more enduring nature of these tools increases the insider status of groups and 
enables them to interact with policymakers on a regular basis (Fraussen et al. 2015). Typically, 
these approaches are used when policymakers aim to consult a narrower and rather well defined 
target group. These approaches are therefore sometimes also described as “targeted”. 

A consultation procedure in the context of a specific policy issue can also involve the combi- 
nation of some of these tools, such as a public internet consultation open to everyone, followed 
by the establishment of an advisory committees that, for instance, involves 20 specific partici- 
pants from academia and civil society. This third consultation approach is labelled hybrid as it 
involves elements of both open (public internet consultation) and closed approaches (advisory 
committee). 


Government Strategies, Bureaucratic Motives, and 
Stakeholder Perspectives 


The previous section outlined the great variety of consultation tools that policymakers have 
at their disposal and conceptually distinguished three types of consultation approaches: open, 
closed, and hybrid. In this section, we shift our attention to the strategies and motives that might 
clarify preferences for certain consultation tools and approaches, addressing these questions from 
the perspectives of policymakers as well as external stakeholders. 


Government Strategies 


To understand variation in consultation approaches, it is important to provide insight into the 
diverse reasons policymakers might prioritize a particular consultation approach above another. 
In recent work, Bali and Halpin distinguish four different strategies policymakers can apply to 
manage the demands of external stakeholders: policymakers can rely on consultation tools to 
routinize, regularize, or generate demands or instead rely on them to impose issues (2021). 
When policymakers aim to routinize demands, they seek to “acknowledge and engage with a 
spectrum of actors, and design strategies to meet these demands” as well as turn “ill-structured or 
chaotic patterns of engagement with organized interests into more routinized forms” (Bali and 
Halpin 2021: 337). Policymakers can routinize demands by setting up and standardizing the use 
of particular consultation tools, such as the standard use of online public consultation (as done 
by the European Commission) or the establishment of expert committees or advisory councils 
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whose composition often remains identical for a longer period of time. In some cases, these 
committees or councils can also have a more ad hoc character as they are set up in response to a 
specific crisis or to address a particular policy challenge that cuts across policy domains. 

Rather than routinizing demands, policymakers can also intend to “regularize” demands. 
The use of these tools ensures policymakers that particular issues automatically return to the 
government agenda at specific intervals, hence creating recurring consultation patterns. The 
examples here are annual budgetary and fiscal calendars or statutory set reviews, which ensure 
government and media attention to these matters on a regular bias. In short, the strategy to 
regularize demands refers to routines and procedures governments establish to ensure a smooth 
policy rhythm and to create a certain level of predictability and control. 

The strategies of “routinizing” and “regularizing” aim to make engagement with external 
stakeholders more structured and predictable and can be considered ways to control, limit, and 
steer policy demands from external stakeholders. Consultation tools related to these strategies 
typically involve a standardized way of selecting and inviting stakeholders and detailed bureau- 
cratic guidelines and procedures for collecting and processing their policy input. A strategy to 
routinize and regularize might lead to a stronger emphasis on closed or hybrid consultation 
approaches. For instance, policymakers could establish an advisory committee or combine pub- 
lic consultation with the formation of expert committees. 

A government strategy can also aim to “generate” policy demands. This relates to the need 
of governments to support, or even create, policy partners, to ensure there is a policy public in 
a specific sector or related to a particular policy issue. Government funding of specific interest 
groups and societal organizations can strengthen (or facilitate the emergence) of a community 
that can support policy programs. While Bali and Halpin in their work focus on the mobiliza- 
tion of possible policy partners, another take on this strategy to “generate” demands would 
focus on the ability of governments to enable external stakeholders to put new issues on the 
government agenda or provide them the opportunity to develop new policy measures in dif- 
ferent institutional settings so that they are less constrained by existing policies and procedures. 

In other words, rather than restricting the space to provide input (for example, by determin- 
ing the scope of the consultation and the specific questions formulated), when seeking to gener- 
ate policy demands, governments maximize the room for policy deliberation and innovation. 
New forms of stakeholder consultation, such as co-design, co-production and collaborative 
platforms, provide relevant examples of this approach. The reliance on those tools is often situ- 
ated within an open or hybrid consultation approach that enables the participation of a wider 
set of participants and, in some cases, also implies the absence of (government) gatekeepers and 
a less central role for public officials. 

The fourth strategy could be considered the extreme opposite of the “generating” approach 
as governments here seek to “impose” issues on the agenda. Here, consultation is nonexistent or 
merely symbolic, as governments strive to “unilaterally install government issue priorities onto 
the agenda”: for instance, via summits or executive statements that enable no (or only a very 
limited) role for external stakeholders (Bali and Halpin 2021: 339). 

In these cases, a government uses its legitimacy (for instance, soon after elections or in 
response to a crisis) “to advance preferred agendas while dismissing the need for public con- 
sultation or engagement” (339). This strategy resonates with a closed consultation approach as 
policymakers here act as gatekeepers focused on limiting and controlling the possible contribu- 
tion and policy input of external stakeholders. 

The ability of government to prioritize particular strategies might also be constrained by 
(domestic) legal requirements (for instance, the obligation to organize public consultations on 
particular topics or the requirement to consult particular advisory bodies), as well as institutional 
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and cultural norms (such as differences in interest intermediation in neo-corporatist and pluralist 
political systems or different traditions in terms of relying on academic experts). 


Bureaucratic Motives 


The interaction between policymakers and external stakeholders is often conceived as an 
exchange relationship, with expertise and legitimacy being key currencies (Bouwen 2004; Tall- 
berg et al. 2015). A similar logic applies to the consultation of external stakeholders in the 
agenda-setting phase. Public officials rely on expertise as they aim to increase the “problem- 
solving capacity” of governments (Van Ballaert 2017: 2, see also Princen 2011). Whereas the 
supply of detailed and relevant policy input is expected to increase the effectiveness of proposed 
policy measures, the involvement of relevant external societal stakeholders is considered impera- 
tive to garner sufficient societal support for policy proposals. A higher societal legitimacy of 
policy proposals is believed to benefit smooth political decision-making processes, as well as 
positively affect accurate policy implementation and compliance. 

Focusing on the role of the bureaucracy, Binderkrantz et al. highlight two “central logics of 
bureaucratic behavior: concerns with securing and enhancing institutional power and bureau- 
cratic autonomy” (2020: 474). These logics highlight the role of consultation within the broader 
political-institutional context. As regards institutional power, Bunea and Thomson, for instance, 
argue that the use of stakeholder consultation may strengthen the agenda-setting and bargain- 
ing power of the government (2015). Focusing on the EU context, they find that extensive 
and open consultations with interest groups in early phases of the policy process increases the 
likelihood of policy success for the European Commission and strengthens its position vis-a-vis 
other EU institutions. 

Binderkantz et al. (2020) link the concept of bureaucratic autonomy to recent work on 
bureaucratic reputation and clarify how an emphasis on distinct aspects of reputation might lead 
to a preference for different consultation practices. They rely on the four-fold distinction made 
by Carpenter between the technical, performative, moral and legal-procedural aspects of reputa- 
tion (2010; for an application in the EU context, see Rimkute 2020). Technical and performa- 
tive reputation can be linked to the earlier-mentioned problem-solving capacity of government 
as it refers to in-house knowledge as well as organizational policy capacities. As expertise is the 
key resource here, Binderkrantz et al. assume that an emphasis on technical and performative 
aspects of reputation will lead to a greater use of closed consultation approaches. If we consider 
moral and legal-procedural reputation, the relative emphasis is on legitimacy. If political institu- 
tions aim to demonstrate those aspects of their reputation, open and hybrid approaches might be 
more suitable since they have the potential to result in “broad consultation involving not only 
interest groups that are usually among the stakeholders of the involved agency, but also allowing 
for the expression of broad, public interest” (2020: 475). 

The central distinguishing feature between open, closed, and hybrid consultation approaches 
involves the accessibility of the consultation procedure. Van Ballaert highlights another key 
dimension of consultation practices: namely, their repetitive nature. Does it concern a one-time 
interaction or repeated exchanges and discussions with the same actors? (2017). This variation 
in the repetitive nature is equally relevant to understanding how consultation practices shape the 
ties between external stakeholders and policymakers as multiple interactions between the same 
actors are likely to lead to a more lasting and stable relationship. 

To understand the origins and dynamics of these long-term interactions, Braun distinguished 
two logics that jointly explain the enduring and often stable nature of policy networks (2013). 
She distinguishes between the logic of anticipatory behavior and habitual behavior. In brief, 
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“the logic of anticipatory behavior refers to maintenance of interactions because of future costs 
associated with ending current interactions” (2013: 6). For instance, it might be costly to end 
existing relationships because there are few alternative stakeholders who can supply relevant 
expertise or legitimacy and since cooperation of particular societal actors and organizations is 
imperative to (future) policy success. While more long-term strategic considerations are at the 
core of this logic of anticipatory behavior, the logic of habitual behavior clarifies how interac- 
tions are shaped by patterns and legacies from the past. Rather than following from specific 
policy objectives or policy needs, interactions are mostly driven by organizational routines, 
which is likely to further solidify existing relationships and thus benefits external stakeholders 
who have already acquired the status of policy insider. 


Stakeholder Perspectives 


From a stakeholder perspective, involvement in the agenda-setting phase is crucial as key deci- 
sions are made by policymakers at this early moment that often cannot be substantially changed 
in the following stages of the policy cycle. This is also the stage of the policy cycle that provides 
most opportunities for shaping the specific content, scope, and focus of new policy proposals. 
If we conceive interactions between policymakers and external stakeholders as an exchange that 
works best when (stakeholder) supply meet (policymakers’) demand, those stakeholders who 
are able to provide critical exchange goods such as expertise and legitimacy are most likely to 
have a listening ear among policymakers. In the following paragraph, we relate this discussion 
to the variety of policy goods that external stakeholders can provide, their central objectives for 
participating in policy consultation, and the use of different consultation approaches. 

It is important to look beyond the classic expertise-legitimacy distinction and consider in 
more detail the variety of policy goods, or policy capacities, that external stakeholders may pro- 
vide. For instance, Daugbjerg et al. highlight not only societal legitimacy and expert knowledge 
but also assistance in implementation, the provision of services to citizens, the mobilization of 
a particular constituency, and the ability to discipline members or ensure compliance within 
a certain profession or industry (2017). These policy goods can be linked to different policy 
capacities of external stakeholders, such as analytical (e.g., expert knowledge), operational (like 
assistance in policy implementation), or political (skills such as societal legitimacy or the ability 
to mobilize a particular constituency). The value of these policy goods and capacities and the 
extent to which they are in high demand among policymakers are likely to be shaped by institu- 
tional factors (such as the policy capacity of government departments, the specific phase of the 
policy cycle, and whether patterns of interest intermediation resemble neo-corporatist or plural- 
ist patterns), as well as more contextual policy-related factors (such as the complexity, salience, 
and conflictual nature of an issue and the number of external stakeholders who participate in a 
specific consultation process) (Kltiver et al. 2015). 

A fundamental question is which consultation approaches or tools are most suitable to acquire 
specific goods. As clarified by Bryson (2013: 27) “specific stakeholders may be involved in dif- 
ferent ways at different steps or phases of the processes”, as the needs of policymakers might shift 
and therefore other consultation approaches become more suitable. For instance, whereas the EU 
Commission relies on expert groups and workshops to acquire detailed policy information, the 
main goal of its online consultations is the involving of a larger set of external stakeholders (Euro- 
pean Commission 2017: 385). From a stakeholder perspective, this also implies that the value of 
policy goods (and participation more generally) will vary across different consultation approaches. 

From a stakeholder perspective, the value of consultation processes is often linked to the 
extent to which these interactions provide opportunities to influence policymakers. This is also 


154 


Consultation Tools and Agenda-Setting 


the dominant focus in many studies, even though the reasons external stakeholders participate 
in policy consultation and value being involved in this early stage of the policy process might 
be much more multi-faceted. For instance, while Lundberg acknowledges the role of political 
influence and the opportunity to shape government public policy as key drivers for stakeholder 
participation in consultation processes, he also highlights role of communicative influence, in 
which stakeholders consider consultations as a means to generate media attention. 

In addition to these external considerations, Lundberg argues that more internal motives can 
be an important motivation for external stakeholders. That is, consultation processes also require 
stakeholders to engage in internal deliberation and discussion with members (or internal stake- 
holders), determine policy positions, and declare public statements (see also Halpin and Fraussen 
2017). More generally, it can even be seen as part of a “civic duty” for external stakeholders, as 
they consider this part of their mission and responsibilities. 

It is important to distinguish between the value of specific consultation tools for exercising 
policy influence and their usefulness for signaling policy preferences (toward policymakers and 
the broader public). While open consultation approaches with a more public and transparent 
character (such as online internet consultation that afterward provide an overview of all input 
to the general public) might be very effective for clarifying the policy position of a specific 
stakeholder to a broad audience, closed approaches (such as expert councils, advisory councils 
or informal direct meetings) are likely to provide better opportunities for policy influence. As 
Binderkrantz et al. argue, participation in open approaches is relatively easy and low cost for 
external stakeholders, and requires no pre-existing relationships with policymakers (2020). In 
contrast, closed approaches often involve considerable investment in terms of policy preparation 
and building and maintaining relationships. Yet these fora also provide opportunities for more 
frequent interactions with policymakers as well as unique opportunities to monitor and discuss 
policy developments. Therefore, we can expect a strong preference among external stakeholders 
for closed consultation approaches as these approaches enable the formation of more enduring 
relationships and limit the competition of other stakeholders who might have different view- 
points. These closed settings with few participants might also bolster more mutual understand- 
ing and trust between external stakeholders and policymakers. 

At the same time, the evaluation by external stakeholders of the legitimacy of consultation 
tools and approaches might be shaped by other considerations and factors. For instance, the extent 
to which external stakeholders can be considered policy insiders or policy outsiders might shape 
their perceptions of particular consultation regime (but see Bunea 2017 for contrary findings). A 
similar dynamic has been demonstrated in research on the legitimacy of participatory processes as 
the perception of individuals of the legitimacy of the procedural features of the process is strongly 
shaped by the outcome and the extent to which the final policy proposal or decision aligns 
with their preferences (e.g., Esaiasson et al. 2019). In that regard, an assessment of losers’ (those 
with different preferences or with less strong ties to policymakers) perception of consultation 
approaches might provide a critical test for their legitimacy (Werner and Marien 2020). 


Consultation and Agenda-Setting: Fairness and Inclusiveness 


When designing consultation processes, determining specific approaches, and selecting particu- 
lar tools, a central point of attention involves the extent to which they create an equal playing 
field among external stakeholders. This notion can be related to the treatment of policy partici- 
pants and the way in which their input is processed, as well as to the open nature of the process 
and the ease of participation. In this final section, we address these two key characteristics of 
consultation processes and their fairness and inclusiveness. 
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Fairness 


The question of the fairness of the consultation process relates to the extent to which participants 
feel that there is an impartial and just treatment of all participants and the input they provided. 

As regards an impartial and just treatment of all participants, open approaches by definition 
score higher in terms of fairness then closed approaches. If we consider, for instance, online 
public consultations, all participants have an equal chance to participate and provide their input 
in a similar way. Yet the treatment of their provided input is likely to differ as some viewpoints 
might align closer with policymakers’ preferences and will therefore be better reflected in pro- 
posed policy. Even if these procedures are highly transparent, there still is a considerable black 
box between the provision of input by external stakeholders and the policy proposals that poli- 
cymakers subsequently put forward based on the variety of input they have received. 

This black box challenge is also critical for the legitimacy of democratic innovations, such as 
mini-publics of citizens, collaborative stakeholder platforms, or cocreation and codesign as par- 
ticipatory tools for agenda-setting. While these tools are often used to “generate” demands by 
external stakeholders rather than “routinizing” or “regularizing” them, if often remains unclear 
to what extent, how, and at what stage of the policy process policymakers need to engage 
with the outcomes of these participatory processes (e.g., Jacquet and van der Does 2021). An 
adequate “coupling” of these initiatives with legislative procedures not only requires particular 
institutional mechanisms but also “requires actors to step outside their comfort zone to build 
new relationships and engage in new spaces with different sets of ideas, actors and rules” (Hen- 
driks 2016: 57). 

The use of these latter tools also often implies a shift in the position and role of public author- 
ities and government officials (Ansell and Gash 2008). While they occupy a central or even 
gatekeeping role in more classic and routinized forms of consultation, such as advisory councils 
and expert committees, they often take up a more facilitative or even neutral role in more col- 
laborative settings. Furthermore, while policymakers often determine the focus or scope of 
consultation in the context of traditional tools (by determining questions from an online con- 
sultation or asking advice of expert committees on specific aspects of policy proposals) and thus 
constrain the opportunity to put new issues on the agenda or formulate policy proposals that 
deviate substantially from the status quo, their ability to shape the agenda and policy discussion 
is reduced as tools put more emphasis on collaboration and deliberation. 


Inclusiveness 


The importance and value of an inclusive consultation process is frequently highlighted. For 
instance, Nabatchi argues that by consulting broadly and involving a diverse set of actors and 
organizations, public officials “give voice to multiple perspectives and different interests, allow- 
ing for more thoughtful decisions that take a broader view of those who will benefit or be 
harmed by an action” (Beierle and Cayford 2002; Sirianni 2009). Public policy also might 
benefit from a more inclusive set of policy participants as this might “generate better projects 
and policies, secure buy-in for decisions, and limit delays, mistakes and lawsuits” (Burby 2003),” 
especially if these processes become more inclusive (Bryson et al. 2013: 28; see also Feldman 
and Quick 2009). 

Findings on the implications of consultation approaches for stakeholder diversity are mixed. 
Some research clarifies how consultation regimes could alleviate inequalities and generate a 
more equal playing field between insiders and outsiders (Bunea 2017). Yet most work points 
in a less positive direction as consultation practices reinforce existing practices and the policy 
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status quo as (also more open) consultation procedures are dominated by established (and often 
economic) organizations that represent business interests. Rather than being a tool for policy 
change, consultation instruments thus become “a primary mechanism of policy reproduction” 
(Howlett and Cashore 2009). 

If we consider the distinct consultation approaches described earlier in this chapter, open 
consultation approaches naturally enable the participation of a larger set of stakeholders than 
closed consultation approaches as participation is “self-selected”, and the number of contribu- 
tions quasi-unlimited. 

The possible relation between consultation approaches and the diversity of participating 
stakeholders is less straightforward. One the one hand, the involvement of more external stake- 
holders in open approaches might lead to a greater diversity of interests that engage with policy- 
makers. Such an approach implies that opportunities to participate are equal and characterized 
by low thresholds. Moreover, there is no gatekeeper who decides who is included and excluded 
from the process. One the other hand, the increase in the number of participating stakehold- 
ers might have a negative relation to diversity. Rather than involving a broader field of actors, 
those stakeholders who dominate closed approaches could also be well represented in open 
approaches, or even become more dominant. In that scenario, those societal groups that suffer 
less from collective action problems and have higher levels of organized representation (such as 
business and professional interests) might be even more numerous and dominant in consultation 
approaches with an open-door policy. 

This last expectation is also confirmed in recent work that examines the use of consultation 
tools by the European commission in the context of 41 regulations and its implications for stake- 
holder diversity, in particular business bias or the extent to which business interests dominate the 
set of organized stakeholders (Fraussen et al. 2020). As regards the applied consultation strategy, 
a purely open approach was rare and only found in 3 regulations. In the majority of cases, the 
EC applies either a closed approach (19 regulations) or a hybrid approach (19 regulations). As 
expected, hybrid approaches, which combine tools related to closed (such as an expert commit- 
tee) and open approaches (internet consultation), attract much larger numbers of external stake- 
holders. Hence, to purely increase the number of involved stakeholders, hybrid approaches seem 
essential. Yet this higher level of engaged external stakeholders does not automatically result in 
a greater diversity of external stakeholders. Instead, closed approaches were characterized by 
higher stakeholder diversity (or less business dominance) than hybrid approach. 

Hence, if we consider the relation between consultation approaches and diversity, we should 
be careful to draw strong conclusions regarding the possible benefits of more open or hybrid 
approaches and not underestimate the role that public officials might (or can) play in designing 
more closed approaches and enabling a more balanced set of participating stakeholders. While 
the role of gatekeeper is often discussed in more negative terms (as they, by definition, exclude 
certain actors and organizations from the policy process and rarely need to justify their choices), 
their possible contribution to achieving a more balanced or equal participation and (better) 
involving overlooked or marginalized constituencies, should not be underestimated and seems a 
promising avenue for further research. 


Conclusion 


In this chapter, we addressed different aspects of the consultation process. We addressed the 
proliferation of consultation tools and distinguished open, closed, and hybrid consultation 
approaches. After clarifying the consultation toolbox that policymakers have at their disposal, we 
examined different government strategies to manage the policy demands of external stakeholders 
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and considered the value of distinct consultation approaches from the perspectives of bureaucrats 
and external stakeholders. As designing consultation process also requires important democratic 
trade-offs, the last section addressed the fairness and inclusiveness of consultation procedures. 

Whereas much scholarly attention has been focused on which societal interests are repre- 
sented in the consultation process, a better understanding of this central component of con- 
temporary governance and its implications for public policy also requires close attention to the 
motives of bureaucrats and stakeholders, as well as their perception of the value and limitations 
of particular consultation tool. Future work would benefit from better connecting the different 
strands of research that have been highlighted in this chapter, bridging the agenda-setting and 
policy formulation phase, and studying the implications of both institutional design choices and 
trade-offs made be external stakeholders and policymakers. 


References 


Ansell, C., & Gash, A. (2008). Collaborative Governance in Theory and Practice. Journal of Public Admin- 
istration Research and Theory, 18(4), 543-571. 

Bali, A., & Halpin, D. (2021). Agenda-Setting Instruments: Means and Strategies for the Management of 
Policy Demands. Policy and Society, 40(3), 333-344. 

Beierle, T. C., & Cayford, J. (2002). Democracy in practice: Public participation in environmental decisions. Wash- 
ington, DC: Resources for the Future. 

Binderkrantz, A. S., Blom-Hansen, J., & Senninger, R. (2020). Countering bias? The EU Commission’s 
Consultation with Interest Groups. Journal of European Public Policy, 28(4), 469-488. doi:10.1080/135 
01763.2020.1748095. 

Bouwen, P. (2004). Exchanging Access Goods for Access: A Comparative Study of Business Lobbying in 
the European Union institutions. European Journal of Political Research, 43(3), 337-369. 

Braun, C. (2013). The Driving Forces of Stability: Exploring the Nature of Long-Term Bureaucracy: 
Interest Group Interactions. Administration & Society, 45(7), 809-836. 

Bryson, J. M., Quick, K. S., Slotterback, C. S., & Crosby, B. C. (2013). Designing public Participation 
Processes. Public Administration Review, 73(1), 23-34. 

Bunea, A. (2017). Designing Stakeholder Consultations: Reinforcing or Alleviating Bias in the European 
Union System of Governance? European Journal of Political Research, 56(1), 46—69. 

Bunea, A. (2020). Stakeholder Consultations. In P. Harris, A. Bitonti, C. Fleisher, and A. Skorkjzr Binder- 
krantz (Eds.), The Palgrave Encyclopedia of Interest Groups, Lobbying and Public Affairs. Cham: Palgrave 
Macmillan. 

Bunea, A., & Thomson, R. (2015). Consultations with Interest Groups and the Empowerment of Execu- 
tives: Evidence from the European Union. Governance, 28(4), 517-531. 

Burby, R. J. (2003). Making Plans that Matter: Citizen Involvement and Government Action. Journal of the 
American Planning Association, 69(1), 33—49. 

Carpenter, D. (2010). Reputation and Power, Organizational Image and Pharmaceutical Regulation at the FDA. 
Princeton, NJ: Princeton University Press. 

Daugbjerg, C., Fraussen, B., & Halpin, D. R. (2017). Interest Groups and Policy Capacity: Modes of 
Engagement, Policy Goods and Networks. In W. Xun, M. Howlett, & M. Ramesh (Eds.), Policy Capac- 
ity and Governance: Assessing Governmental Competences and Capabilities in Theory and Practice (pp. 243— 
261). Cham: Palgrave Macmillan. 

Esaiasson, P., Persson, M., Gilljam, M., & Lindholm, T. (2019). Reconsidering the Role of Procedures for 
Decision Acceptance. British Journal of Political Science, 49(1), 291-314. 

European Commission. (2017). Better Regulation “Toolbox”. Brussels: European Commission. 

Feldman, M. S., & Quick, K. S. (2009). Generating Resources and Energizing Frameworks Through 
Inclusive Public Management. International Public Management Journal, 12(2), 137-171. 

Fraussen, B., Albareda, A., & Braun, C. (2020). Conceptualizing Consultation Approaches: Identifying 
Combinations of Consultation Tools and Analyzing Their Implications for Stakeholder Diversity. Policy 
Sciences, 53(3), 473-493. 

Fraussen, B., Beyers, J., & Donas, T. (2015). The Expanding Core and Varying Degrees of Insiderness: 
Institutionalised Interest Group Access to Advisory Councils. Political Studies, 63(3), 569-588. 


158 


Consultation Tools and Agenda-Setting 


Halpin, D. R., & Fraussen, B. (2017). Laying the Groundwork: Linking Internal Agenda-Setting Processes 
of Interest Groups to Their Role in Policy Making. Administration & Society, 51(8), 1337—1359. 

Hendriks, C. M. (2016). Coupling Citizens and Elites in Deliberative Systems: The Role of Institutional 
Design. European Journal of Political Research, 55(1), 43—60. 

Howlett, M., & Cashore, B. (2009). The Dependent Variable Problem in the Study of Policy Change: 
Understanding Policy Change as a Methodological Problem. Journal of Comparative Policy Analysis, 
11(1), 33-46. 

Jacquet, V., & van der Does, R. (2021). Deliberation and Policy-Making: Three Ways to Think about 
Minipublics’ Consequences. Administration & Society, 53(3), 468—487. 

Kliiver, H., Braun, C., & Beyers, J. (2015). Legislative Lobbying in Context: Towards a Conceptual Frame- 
work of Interest Group Lobbying in the European Union. Journal of European Public Policy, 22(4), 
447—461. doi:10.1080/13501763.2015.1008792. 

Princen, S. (2011). Agenda-Setting Strategies in EU Policy Processes. Journal of European Public Policy, 
18(7), 927-943. 

Rimkutė, D. (2020). Building Organizational Reputation in the European Regulatory State: An Analysis 
of EU Agencies’ Communications. Governance, 33(2), 385—406. 

Sirianni, C. (2009). Investing in Democracy: Engaging Citizens in Collaborative Governance. Washington, DC: 
Brookings Institution Press. 

Tallberg, J., Dellmuth, L. M., Agné, H., & Duit, A. (2015). NGO Influence in International Organizations: 
Information, Access and Exchange. British Journal of Political Science, 48(1), 213-238. 

Van Ballaert, B. (2017). The European Commission’s Use of Consultation during Policy Formulation: The 
Effects of Policy Characteristics. European Union Politics, 18(3), 406—423. 

Werner, H., & Marien, S. (2020). Process vs. Outcome? How to Evaluate the Effects of Participatory 
Processes on Legitimacy Perceptions. British Journal of Political Science, 1-8. Doi:10.1017/s000712342 
0000459. 


159 


13 


TOOLS FOR MANAGING 
THE MEDIA 


Monitoring, Branding and Messaging 


Alex Marland 


Communication is a crucial component in the success or failure of public policy, including 
agenda-setting. As the media and technological environment evolves, so, too, must politicians 
and governments, which means packaging information to meet the way the news media works 
and the way audiences receive and process information. The many disparate voices in a digital 
world have increased the risks of a communications mistake and augmented the strategic value 
of controlled messaging. This control can take the form of branding whereby all information is 
coordinated to reinforce central messages that resonate on an emotional level and foster loyalty 
and which offer time and cost efficiencies but which emphasize select aspects of a policy while 
obscuring others. This chapter looks at some of the public-facing communications activities by 
government to sell public policy. It examines some internal strategic planning templates used in 
Canada, where branding and message management are key facets of public policy announce- 
ments. While public sector branding is generally in the public interest, it is a form of informa- 
tion management that bears monitoring. This chapter reveals some of the hidden aspects of 
communications planning in government. It begins by situating why public officials want to 
control information and synthesizes literature about public sector branding. Most of the chapter 
features a case study of the government of Canada, and discusses how internal tools are used 
to coordinate the repetition of strategic messaging across multiple communications channels. 
Examples of communication tools are identified to illustrate the political interest in spinning 
information, centralized communications planning and the onset of the shift to digital visu- 
als, social media and storytelling. Special attention is paid to key message planning tools which 
involve public servants strategizing how to optimize spin, generate positive publicity and rein- 
force brand messaging. 


Introduction 


Politicians and political staff obsess over trying to control communications. Sophisticated news 
management in government dates to at least the Second World War (Tulloch 1993), but nowa- 
days, the desire to generate praise and avoid criticism extends beyond news coverage to preoc- 
cupation with social media and public opinion polls. The relentless push for positive publicity 
and the integration into governance of election campaign strategies and tactics are so persistent 
that governments are said to be permanently campaigning (e.g., Cook 2002; Needham 2005). 
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As partisans apply a command-and-control hierarchy to government, and as they infuse new 
communications-focused processes, they demand competent communications management at 
all stages of the public policy cycle. The unrelenting drive among partisans to win political com- 
petitions shows no signs of letting up in a hyper-mediated public environment where a minor 
slip-up can go viral and derail a policy agenda. This political fixation with communications 
victories is magnified when combined with the public service’s own efforts to manage informa- 
tion in a fast-tempo mediascape that blends journalists, political actors and ordinary citizens in 
real-time public interactions without boundaries of place. 

Modern governments are preoccupied with information management, including branding 
(Eshuis and Klijn 2012) and self-promotion (Deacon and Monk 2002). Branding emerged as a 
marketing tool to differentiate commodities such as coffee and soap through the use of logos, 
colourful packaging, contests and publicity that resulted in consumers willing to pay a price 
premium for their preferred brand (Marland 2003). It evolved into fostering brand preference by 
communicating images and emotions instead of emphasizing details like car advertising, which 
promotes lifestyle and aspiration. 

Today, branding is a monolith that permeates organizational culture. It unifies all commu- 
nications to promote desired key messages, construct a desired image and reach target audi- 
ences in an efficient manner that saves money, reinforces overarching corporate values, generates 
goodwill, builds emotional attachments and instills loyalty (Marland 2016; Wood 2000). While 
most, if not all, leaders of Western liberal democracies are attuned to the importance of staying 
“on brand” (e.g., Scammell 2015), it would be misguided to assume that the appeal of branding 
among political parties is what has spurred its adoption within public administration, which has 
its own reasons for using branding. As well, branding does not exist in a vacuum of related phe- 
nomena, such as the packaging of politics (Franklin 1994) or new political governance (Aucoin 
2012), and like other forms of communications management, it can obfuscate what strategists 
do not want audiences to see by employing spin and propaganda (Bertilsson and Rennstam 
2017). A notable difference from other forms of strategic communications is that the simplic- 
ity of brand messages combines financial savings with nurturing a flock of devoted supporters. 

Branding in government appeals to the political executive, political staff and senior public 
servants alike, although there can be tension about its execution. They generally recognize the 
competitive and financial benefits of differentiating products with country of origin markers, 
of creating tourism campaigns that turn on the branding of a place, and of branding public 
policy (e.g., Alserhan 2013; Valaskivi 2013). They periodically collaborate to develop a new 
corporate identity for the government, whose visual standards and logo are enforced by staff 
throughout the public service. Branding is also used by governments to foster public trust and to 
forge stronger relationships with internal and external stakeholders (Letjerholt, Biedenbach and 
Hultén 2019). How politicians and governments manage some forms of communication is well 
documented. For instance, relatively observable phenomena include the role of press secretaries 
to drive the news agenda or ways that governments reinforce the leader’s authority (Heffernan 
2006). Less well understood are the internal practices in government that bring together dispa- 
rate messages to present a unified voice and image as part of a branding strategy. 

What tools do governments use to coordinate brand messages? What public administration 
processes are used to promote conformity and ensure alignment with centralized messaging? 
Surprisingly little documentation exists about internal tools within public administration to 
manage communications. There are two main types of communications templates within gov- 
ernment that encompass front-end and back-end activities (Gaber 2000; Howlett 2009). One 
type is publicly visible, such as a news release or website content, while the other is below the 
line and performs an internal organizational function, such as a planning document to roll out 
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the announcement of a new policy (Marland 2017a). There is little descriptive research about 
below-the-line media management by governments around the world, with the exception of 
practitioner manuals that offer insider accounts, including How to be a Spin Doctor (Richards 
2016) and Professional Communications in the Public Sector (Glenn 2014). 

In this chapter, I synthesize media management tools used in public administration, paying 
particular attention to internal planning functions within Canadian federal and provincial gov- 
ernments and how this functions as part of branding. This synthesis draws on a review of internal 
documents obtained in 2016 by placing requests with departmental communications staff and 
filing access-to-information requests (see Marland 2017a, 2017b). Readers should be mindful 
that communications strategies and tactics are constantly evolving in response to changes in 
technology, journalism and society, including innovations in the private and public sectors, as 
well as politicians and political staff importing new approaches from the campaign trail. The 
Canadian case informs broad insights into the significant planning exerted within public admin- 
istration to control information to achieve optimal publicity and a keen political interest in tying 
messages to the master brand established by those at the top of the government hierarchy. 

Being mindful that the study of a single country constitutes a snapshot in time in a single 
jurisdiction, I summarize processes to enable generalizations that might apply to other coun- 
tries, particularly those using the Westminster system of government (e.g., Aucoin 2012; Gaber 
2000). But before examining how governments manage the media, we need to consider the 
news ecosystem they operate in, which is a hybrid of traditional mainstream media and digital 
media. 


The News Media Ecosystem 


Relationships between politicians and journalists are fraught with tension in Western liberal 
democracies. The nineteenth-century era of the party press, when newspapers were blatantly 
partisan and sometimes owned by elected officials, gave way to a commercialized media envi- 
ronment in which government executives gave inside scoops in return for laudatory coverage. 
The television age in the mid-twentieth century hastened the focus on political leaders, includ- 
ing their presentation and personality, while the Watergate scandal in the late 1960s transformed 
press galleries from reporter lapdogs who built friendships with political insiders into journalists 
aspiring to be agitators who mistrust authority figures. 

The media’s competitive interest in behaving as a so-called fourth estate grew as the speed at 
which news and information travels intensified with technological innovations such as photo- 
copiers, 24/7 news channels, fax machines, cheaper long distance telephone fees and cellphones. 
Throughout, image management was at the fore, with political leaders constantly looking for 
ways to generate the most positive impressions of their public self and policies. 

In this century, information communication technologies have exponentially increased the 
volume and speed at which news travels, including from the general public, while the media 
gatekeepers who decide what is newsworthy have seen their role diminish. At first, politicians 
and public servants were preoccupied with posting text on static web pages. As internet band- 
width expanded and digital cameras became the norm, this evolved into more dynamic experi- 
ences involving graphics, photos, audio and video. 

The emergence of social media and smartphones brought more diverse voices into political 
conversations while simultaneously fracturing audiences into information silos that are filled 
with like-minded views, or “echo chambers,” including among journalists who amplify divisions 
(Usher, Holcomb and Littman 2018). In the digital age, information travels so quickly that there 
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can be multiple news cycles per day, with a hybrid of traditional and newer media combining 
to create a “political information cycle” that features journalists interacting with citizen proto- 
journalists as they compete to be the first to break news online (Chadwick 2011) and react to 
audience online feedback (Lee and Tandoc Jr. 2017). 

Governments were forced to adapt. The wheels of public administration are notoriously slow, 
resistant to change and painstaking, but the public service was confronted with a public culture 
that wanted two-way engagement in real time and political masters who were under pressure to 
divulge error-free information in a manner that reinforced the government’s message themes set 
by the political centre. In a digital communications environment, there is a greater need to pack- 
age information in an interesting, simple and creative manner in order to attract journalists’ and 
audiences’ fleeting attention. At all times, government personnel must be mindful that pundits 
and opponents will seize the opportunity to pounce on a misstep. 

The fear of error, combined with the desire to promote a singular narrative, has driven the 
prevalence of scripting in politics and government. Ministers are advised to stick to their talking 
points, to deliver canned speeches and to avoid answering controversial questions. Social media 
posts rehash key messages and present staged images. In lieu of risky interviews, journalists might 
be asked by a communications staffer to submit enquiries by email so a written response can 
be crafted by multiple people to skirt thorny topics. In response, professional journalists might 
ignore supplied or repeated messaging, which can result in news stories that cause political 
actors discomfort and fortify their impressions that the media cannot be trusted. In other cases, 
codependency persists, such as when journalists report inside information without identifying 
their sources while less industrious and time-strapped reporters get in the habit of quoting from 
social media posts. Often, only well-resourced journalists have the time and space to perform 
investigative journalism (Boswell and Corbett 2016). 


Government Advertising 


To avoid media controversy, governments control the message by buying advertising on plat- 
forms that they believe will reach target audiences. Advertising is a crucial element of brand 
building because it allows the sponsor to direct all facets of the message and how it is com- 
municated, and the controlled images can resonate on an emotional level. Advertising plays a 
considerable role in publicizing public policy, ranging from a public health information cam- 
paign through to tourism ads and annual reminders about filing taxes. However, it is expensive, 
particularly on television, whereas social media ads offer greater precision at a more economical 
price. More ominously, government advertising is subject to taking on partisan slants and allega- 
tions of inappropriate use of state resources for political gain. 

The potential for government ads to imbue partisan tones results in rules and processes. In 
Canada, the federal government has a two-stage oversight mechanism, requiring that propos- 
als for large advertising campaigns be reviewed by a non-profit organization that administers a 
national code of advertising standards. The review criteria focus on whether a proposed adver- 
tisement is considered to be objective, is free from political party identifiers or bias and does 
not blatantly use a political party’s colour scheme (Government of Canada 2020). Other criteria 
include ensuring that politicians do not appear and that certain public policy initiatives or pro- 
posed trade agreements are not advertised until approved by Parliament. A comparable mecha- 
nism in Ontario, the country’s largest province, inspired the federal oversight model, although 
frustrated politicians have since watered it down. Smaller provinces have few if any formal rules 
to guide how government advertising may be used. 
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Media Management: Public Communications Products 


Outward-facing communications products are relatively consistent in Western liberal societies. 
To improve the potential for news coverage, public relations (PR) activities need to respond to 
how journalists perform their work, which means recognizing the pressures they face to file 
news stories quickly, efficiently and at a low cost with minimal effort. Publicists are therefore 
incentivized to package information to accommodate the on-the-job realities of media eco- 
nomics. For example, instead of securing an interview or filing access-to-information requests, 
some journalists investigate stories while sitting at their desks by perusing what is trending on 
social media and conducting online research for background information, which increases the 
strategic importance of managing online content. News outlets that allocate fewer resources to 
original journalism, particularly small media outlets, might rely on news releases for content or 
publish supplied digital photographs (Marland 2012). Accordingly, an organization that posts 
quotes from key spokespersons on Twitter about a topical issue, emails a news release written 
in the manner of a news story or posts professional photographs online increases the likelihood 
of circulation via the news media. The uptake of these information subsidies is so extensive 
that one British study concluded that a majority of news stories avail of free content and that 
only a minority of news reports feature independent journalistic activities (Lewis, Williams and 
Franklin 2008). 

Many media management tools are generic and are creatures of corporate PR. Media adviso- 
ries are issued to announce the details of an upcoming news conference, photo op or other event 
that reporters might want to attend. Ministers deliver speeches by drawing on speaking notes 
that might be shared with the press in advance on a “check against delivery” basis. The written 
word includes news releases, backgrounders, frequently asked questions and fact sheets, while 
digital-only content features multimedia galleries of high-quality photos and video in a style that 
mimics photojournalism and is suitable for reproduction. Partisan slants are often embedded; the 
least politicized PR products are public alerts and notices that provide factual information about 
public safety. All this responds to the norms of how the media behaves, which academics refer to 
as “media logic” and “mediatization”: namely, journalists acting as gatekeepers who filter infor- 
mation, decide what has news value, and apply their own values and biases (Strömbäck 2008). 

Countries such as Canada, Germany, Spain and the United Kingdom have formal rules to 
help ensure that information about government policies is communicated in an impartial man- 
ner to the media (Sanders, Crespo and Holtz-Bacha 2011). But there are limits: interactions 
between journalists and partisan spokespersons such as ministers are not bound by the same 
limitations as non-partisan public servants are, nor should they be. It falls to political representa- 
tives to stay on script and reframe debate. They engage in rebuttal, set and drive the news agenda 
and plant stories, or perhaps they might create diversions, float policy trial balloons and gener- 
ally manage public expectations (Gaber 2000). But how, then, does the public service engage in 
media management that is hidden from public view? 


Media Management: Internal Planning Tools 


It is the below-the-line planning tools that reveal the strategic thinking that influences what pol- 
icy information is released and how it is packaged. Back-and-forth discussion between the head 
of government, ministers, political staff, senior public servants and communications personnel is 
simplified by using standardized templates that ensure all relevant details are considered. Every 
government uses different internal forms, and invariably a government tweaks and updates its 
innumerable templates. What is constant is that considerable human resources are allocated to 
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mapping out how public policy will be announced, how it will be promoted, and how media 
enquiries and social media discussion will be managed. 

A branding ethos is pervasive in the government of Canada, making it an exemplary case 
study. A strategic planning template introduced by the Conservative government led by Prime 
Minister Stephen Harper from 2006 to 2015 exemplified how the Prime Minister’s Office 
(PMO) strives to police message discipline and manage visual optics. A “message event proposal” 
form introduced by the Harper PMO began as an election campaign tool to plan for the leader’s 
media events and was introduced into government, where it went through various iterations (for 
examples, see Marland 2016: 400-404, 2017b: 39-40). The template required that line depart- 
ments supply copious details about a planned public announcement or media event so that the 
PMO and its public service counterpart, the Privy Council Office (PCO), could play a central 
coordinating role. The message event proposal required that a minister’s office and public ser- 
vants bundle the event into one of a handful of branded message themes, such as “creating and 
protecting jobs and opportunities” or “protecting Canadians” to receive clearance from central 
agencies. Departments were obligated to identify key messages; outline the visuals, including 
the backdrop and speaker’s podium sign; and consider newsworthy angles. That the ensuing 
Liberal government led by Justin Trudeau pledged in 2015 to do away with the central over- 
sight, and yet, nevertheless, departmental templates persisted, including some which repurposed 
the notorious form, speaks to the institutionalization of communications planning documents. 
The templates are a facet of public administration in a similar way that memoranda to cabinet, 
briefing notes and other repetitive information assemblies are necessary for efficient government 
operations, except that communications processes are more fluid. 

A myriad of policies and protocols within the government of Canada promote branding. Chief 
among them is the Federal Identity Program (FIP), which is a corporate image policy set by the 
Treasury Board that has passed through multiple iterations since the program’s creation in 1970 (Way 
1993; Marland 2016). The FIP has strict guidelines about the use of visual identity, and it restricts 
departments from creating their own logos. Brand coherence is evident through the requirement 
that government visuals, including the exterior of its buildings, prominently display the iconic Can- 
ada wordmark that comprises “Canada” in black typeface and a red maple leaf flag situated on top 
of the final letter. Each department is required to designate an FIP coordinator to ensure compli- 
ance, although certain agencies, boards and commissions are exempt from the program. The FIP is 
a branding leviathan that sets a culture tone throughout Canadian government that line departments 
are beholden to the centre and must follow a “whole of government” approach to communications. 

Choosing the timing of a government policy announcement or media event is reminiscent 
of air traffic control in that there must be centralized monitoring and coordination to avoid 
conflicts. In Canada, ministers require approval of the PMO to travel outside the capital city 
of Ottawa. Both the PMO and PCO maintain communications calendars that map planned 
ministerial events along with other activities such as public holidays, leaders’ summits, tabling 
of reports of officers of the legislature, events planned by opponents and interest groups and 
so forth. Where possible, government media events are slotted into a message of the day and 
grouped into a weekly theme. Thus, policy announcements involving women might be delayed 
or bumped up to align with International Women’s Day, or policies concerning young families 
might be timed to coincide with children heading back to school after summer vacation. The 
bundling of messages in this way increases the ability of the government to sustain an over- 
arching brand as compared with communications delivered in a hodgepodge fashion. Central 
coordination also avoids the potential for a high-profile policy announcement to overshadow 
another potential good news story or get lost amid external events. Awareness of the external 
political environment is needed to address the possibility that an announcement could be in 
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poor taste, such as if news broke that the government was changing how morticians were regu- 
lated at the very moment that the public was mourning a human tragedy. 

Considerable effort is now steered towards digital communications, and the extent to which 
politicians and public servants attempt to micromanage social media is startling, given that they 
work to create an illusion of being impromptu and authentic. In Canada, communications 
process templates reflect the individual preferences of government departments. Preparing for 
a public announcement can range from filling out a narrow event planning checklist to draft- 
ing a broad strategic communications plan. When journalists request an interview, a staffer 
might complete a form to document relevant details and possibly draft suggested media lines for 
approval. Staff develop and maintain social media plans that encompass how preplanned posts 
and images will be rolled out. An additional element with the government of Canada is that all 
communications must be bilingual, so time is needed to ensure that English can be translated 
into French and vice versa. As a result, a single tweet can be crafted and approved by a dozen or 
more personnel and prepared weeks in advance. Sometimes a communications snafu occurs, and 
an awkward turn of phrase or political insensitivity that is mocked online can lead to a retraction 
or explanation by the minister. The public embarrassment is held up as further evidence that 
communications planning and oversight are necessary. 

The strategic and tactical impetus for communications planning is sometimes spelled out in 
templates so that public servants have greater awareness of the need to give significant thought 
to how they will provide the media or public with information. Plans for publicizing major 
policies, programs and initiatives are organized much like a corporate document. One Cana- 
dian department’s template begins with a summary of what is being communicated and how 
it is linked with government priorities, notably the budget (Employment and Social Develop- 
ment Canada 2016). Public servants are advised that communications objectives require thinking 
about ways to raise awareness of a program or policy and what action audiences should take. 
Isolating target audiences entails, in part, considering their online habits and which social media 
influencers can be reached to spread the message. Knowing audiences’ current perception of the 
issue, identifying communications risks and opportunities, establishing an awareness of trends 
and stakeholder positions, identifying what is being said in the media and estimating how an 
announcement is likely to be publicly received are among the many strategic considerations. 
Increasingly, communications personnel are encouraged to include data points in their messag- 
ing because statistics and milestones add credibility to broader messaging. Having established 
these basics, this particular template then moves into the strategic approach to getting the mes- 
sage out, including digital communications, media outreach and stakeholder engagement. 

Developing key messages is where public policy transforms from detailed minutiae into high- 
level themes and emotive communication. Concision is prized. In the template example here, 
public servants are asked to create “three to five key messages that align with your objectives and 
capture the essence of the initiative.’ They are instructed to “keep the messages concise and in 
plain language” and to “work alongside your program colleagues to ensure messages are simple and 
easily understood, particularly if describing a complex or highly technical topic.” The foundation 
of branding appears in instructions urging unity of message. In this template, communicators are 
asked to “keep in mind the government’s other messages related to this issue, and ensure consis- 
tency across all products.” After crafting messages, the department is requested to list types of public 
communications products and describe how the success of the public rollout will be measured. 

Considerable planning goes into a government’s social media. Canadian departments maintain 
social media implementation plans or some such variation. For example, in 2016, one depart- 
ment’s public servants considered the business drivers of the proposed online approach and how 
it aligned with a broader social media strategy (Agriculture and Agri-Food Canada 2016). They 
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were asked to justify the choice of platform by articulating the target audience and identifying 
what accessibility options are available and what terms of use apply. The form then requested 
details about resources to mobilize the online message, including the need to identify which staff 
will work on the social media plan and at what intervals, along with what financial resources will 
be required. A chart of roles and responsibilities followed to identify who will draft content, edit 
it, approve it and publish it. Once published, someone needs to administer, monitor and respond 
to the account, particularly if issues arise that require swift management. As part of this particular 
template, the department required a risk management plan to mitigate potential problems, with 
additional planning resulting from the proposed use of social media platforms that are not on a 
preauthorized list. Next, management protocols must be considered. How will the department 
engage with audiences online? What protocol will it follow in the event of contentious issues? 
What are the voice and style of the account? Who will have access to running it? What is the 
plan for ensuring continuous updating? A further factor is performance management to measure 
the success of the account relative to its objectives. Then, a communications strategy must be 
prepared to “align key messages communicated through the official social media account with 
messages delivered through other departmental channels” (Agriculture and Agri-Food Canada 
2016). A phasing-out plan follows to establish how the social media account will be phased out 
and closed. All this planning must be approved by the departmental head of communications. 
Government personnel are evidently concerned about the pitfalls of maintaining a social media 
presence. According to a third Canadian department, the objectives of managing risk in social media 
interactions include ensuring that the department engages in a timely manner, maximizes the vis- 
ibility and ranking of its social media channels, and serves the public effectively. To accomplish this, 
it maintains a chart to assign tasks to designated public servants to manage likes, retweets and ad hoc 
messages which can escalate to involving senior management (Table 13.1). This is one of countless 
internal planning documents used within governments worldwide to manage the message. 


Table 13.1 Social Media Engagement Delegated Authority Chart 


Level Social Media Content Type Delegated Authority 


1 Likes Partner/stakeholder content Social Media Advisor 
Call to action responses (e.g., photo 
contest, poll, questions, etc.) 


2 Retweets/shares Partner/stakeholder content Strategic Communications 
Key social media influencers Manager or Manager of 
Web and New Media 

3 General user questions Responses to questions using Strategic Communications 
preapproved messages (media lines, Manager 
Q&As, existing web content, etc.) 

4 General user questions Questions and content outside the Associate Director 

Retweets/shares scope of Level 2 or 3 (such as content General or delegate 


not originating from an identified 
stakeholder/partner or influencer) 


5 Snapshot Weekly, advance content 
6 Ad hoc messages Messages which were not anticipated in 
the snapshot process 
7 As required by Associate Any social media content or question Director General or 
Director General as needed delegate 


Source: Canadian Heritage (2016) 
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3 bDNJ pa APMIS 
Building M/ynavut Together 


Nunavuliuqatigiingniq 
Bâtir le Nunavut ensemble 


Nunavut 


Figure 13.1 Government of Nunavut Logo and Tagline 


Source: © Government of Nunavut. Reprinted with permission 


It is unclear to what extent branding is top of mind for the public servants who fill out these 
templates. In this Canadian case study, the word brand almost never appeared in the collected docu- 
ments, with the key exception being sub-national governments that create brand standard manuals 
similar to the FIP. Documentation for the government of Nunavut’s visual identity program is 
particularly enlightening. That northern territory’s manual outline brand compliance expectations, 
such as how to use the government’s logo (Figure 13.1), what its design standards are for the logo 
and wordmark, how to use the government’s tagline and examples of improper use. 

The Nunavut manual explains what branding is and establishes that it is a prism for all gov- 
ernment policy decisions: 


The [government’s] brand is broader than its visual identity, more than a signature 
or symbol. Our brand is the intangible sum of the government’s attributes: its name, 
values, offerings, people, history and reputation, and the way it is experienced and 
promoted. ... The brand positioning provides a lens through which every substantive 
decision within the [government] should be made. 

(Government of Nunavut 2016: 2) 


It goes on to explain that the visual identity of a polar bear logo helps differentiate the ter- 
ritory from other sub-national governments; assist citizens with easily recognizing government 
programs, services and facilities; and reinforce the government’s reputation. The benefits of 
branding, it says, are that the government develops “effective tools to communicate consistently 
and clearly to all audiences, at a number of levels, while ensuring that all communications rein- 
force our reputation, wherever we go.” By comparison, mentions of branding were fleeting in 
the other collected documents, such as one province’s corporate communications policy, which 
referred to the importance of brand coherence (Treasury Board of Nova Scotia 2016). 


Conclusion 


The synchronization of all communications is a growing trend that has considerable benefit 
and is considered essential in a multi-platform media environment. Politicians and political staff 
adopt corporate marketing practices during election campaigns and then import successful prac- 
tices into governance. The public service also employs branding, most notably through the uni- 
fication of messaging and visuals under a single corporate identity umbrella. The considerable 
cost and time savings that result from communications consistency, combined with improved 
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ability to reach audiences in a more poignant manner, understandably make branding a strategic 
priority for public administration. However, this brings the potential for political interference 
and obfuscation of information that can weaken source credibility and harm a professional pub- 
lic service’s creed of impartiality. 

Government branding comprises visible communications and the internal processes that 
drive those outward-facing interactions. Advertising is the most dynamic aspect of a branding 
campaign because the sponsor controls all strategic facets, ranging from the creative design to 
the media buy. But advertising is expensive. Whether advertising is used or not, the news media 
plays a formidable role in reporting information about government activities and public policy, 
and journalists in modern liberal democracies often file confrontational reports. Governments 
package information in a manner that increases the likelihood of the media abandoning a criti- 
cal filter in its reporting. Press conferences, photo ops, tweets and news releases are part of the 
canon of PR tactics to fulfill the media’s interest in low-cost and timely information supplied in 
a user-friendly format. In short order, social media has become the preferred medium for offi- 
cial content, and consequently, each post can require the involvement of many public servants. 

Considerable work goes on behind the scenes within government to influence media cov- 
erage and control their agenda or at least manage them; by extension, this involves promoting 
interconnected messages that are a hallmark of branding and communications control. Govern- 
ments are large, complex organizations that are prone to using templates to assemble and coor- 
dinate many perspectives, so it stands to reason that they do so with communications planning. 
Those that embody transparency make publicly available the templates of their internal planning 
tools, such as memoranda to cabinet or briefing notes. Conversely, communications templates 
can be difficult to locate, with the notable exception of those related to emergency planning, 
which require urgent nonpartisan coordination across multiple public actors. It seems likely 
that these will remain hidden instruments because making internal templates publicly available 
would subject a government to criticism for attempting to manipulate information and for pri- 
oritizing some audiences over others. As well, while a cabinet memorandum is relatively static 
and centralized, by comparison, the communications ecosystem is subject to constant change; 
thus, government departments must continually revise their templates. 

There is a saying in government that good communications cannot sell bad policy, but bad 
communications can sink good policy. Reinforcing central messages and mitigating the potential 
hazards of derailing a policy initiative make branding even more appealing to public servants and 
politicians. While the use of branding in the public sector is generally in the public interest, it is 
a form of information management that bears monitoring. 
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Policy formulation is an important but imperfectly understood stage of the policy cycle. Chal- 
lenging to conceive and study without thinking in terms of tools — arguably more than for 
other policy stages — there are many individual literatures that seek to promote and/or inform 
the use of specific policy formulation tools. With the emergence of ever-more-complex policy 
problems, a fresh wave of interest in more sophisticated policy formulation tools has been gen- 
erated as well, such as scenarios and computer-based forms of modeling. However, the policy 
analysis literature is relatively silent on how, why, when, by whom, in what settings and with 
what effects the various tools are used in practice. To that end, we propose a new definition and 
typology of tools and offer a means of reassembling the field around an analytical framework 
focused on actors, venues, capacities and effects. To understand these tools fully, we argue that 
policy researchers must view them in the context of the broader activities and processes of policy 
formulation. The general aim here is to investigate what can be gained by bringing the study of 
policy formulation tools back into the mainstream of public policy research. 


Introduction 


As policy researchers and analysts are becoming more interested in policy formulation — partly 
because of the rise to prominence of ever-more-complex problems that defy standard policy 
remedies (Howlett et al. 2014) — there is a growing belief that it may constitute the final ‘missing 
link’ (Hargrove 1975) in policy analysis. If the agenda-setting stage in the policy cycle is essen- 
tially concerned with identifying where to go, policy formulation is all about how to get there 
(Hill 2009, p. 171). Arguably one of the most poorly understood of the policy process stages, 
policy formulation is a vastly different activity to policy implementation. It is an important phase 
devoted to ‘generating options about what to do about a public problem’ (Howlett 2011, p. 29) 
and is inherent to most, if not all, forms of policymaking. In this chapter, we investigate what 
might be gained by bringing the study of policy formulation tools back into the mainstream of 
public policy research. A new definition and typology of tools are proposed, as well as an analyti- 
cal framework, as a means of reassembling the field. 

If policy formulation is ‘a process of identifying and addressing possible solutions to policy 
problems or, to put it another way, exploring the various options or alternatives available for 
addressing a problem’, then developing and/or using policy formulation tools is a vital part of that 
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process (Howlett 2011, p. 30). We suggest that, much more than for other policy stages, it is very 
hard to conceive of policy formulation, let alone properly study it, without thinking in terms of 
tools. Based on Dunn (2004), these include tools for forecasting and exploring future problems 
using scenarios, tools for identifying and recommending policy options (e.g., cost-benefit, cost- 
effectiveness and multi-criteria analyses) and tools for exploring problem structuring or framing 
(e.g., brainstorming, boundary analysis and argumentation mapping), to name but a few. 

In recent years, the number of potentially deployable policy formulation tools has expanded 
massively. They include types that may be considered to fall into both positivist and post-pos- 
itivist categories, with the latter inspired by critiques of the role of technocratic analysis and a 
concern to address subtle influences on the content of policy, such as discourses and ideologies 
(Fischer 1995). Yet the policy tools and instruments literature remains stubbornly fixated on 
implementation instruments. And while there are many individual literatures that seek to pro- 
mote and/or inform the use of specific policy formulation tools, the policy analysis literature is 
relatively silent on how, why, when, by whom, in what settings and with what effects the various 
tools are used in practice. To the extent that they devote attention to formulation as a specific 
stage in the policy process, most textbooks frame it around understandings of processes, interests 
and expertise. 

In many ways, the limited academic treatment that policy formulation tools have received in 
the period following the Second World War is symptomatic of a wider division in policy analysis 
between those doing policy research and those engaged in policy practice. For reasons explored 
more fully later in this chapter, when it comes to policy formulation tools, practice has argu- 
ably run well ahead of research. In this chapter, we seek to bring these two wings of the policy 
analysis community into a closer dialogue. 

The remainder of the chapter is divided as follows: the second section takes a step back by 
examining the main actors, processes and venues of policy formulation in a general sense. The 
third section charts the turn away from policy formulation tools in mainstream public policy 
research and explores some of the reasons interest in policy formulation has recently undergone 
a renaissance. It also scours the existing literatures to explore the development of the several 
policy formulation tools that could, in principle, be used in these venues. Section 4 explores the 
analytical steps that will be needed to reassemble the various literatures into a more coherent 
sub-field of policy research, revolving around a series of common foci. To that end, we propose 
a new definition and typology of tools and offer a means of reassembling the field around an 
analytical framework focused on actors, venues, capacities and effects. 


Policy Formulation: Actors, Processes and Venues 


Actors: Who Are the Policy Formulators? 


It is generally recognized that policy formulation is a critically important but relatively inscru- 
table stage of the policy process (Wu et al. 2010, p. 47), with many different actors interacting, 
often under intense and focused political pressure from special advisors, lobbyists and interest 
groups. As we shall see, this creates a distinct set of challenges for those who want to study the 
use of the tools or those who wish to design and/or promote them. 

Some specific studies of policy formulation have sought to offer a detailed stock take of the 
different policy analysts who are typically involved (Howlett 2011, p. 31). Together, these actors 
are often said to constitute a policy advisory system, comprising decision-makers (chiefly politi- 
cians), knowledge producers and/or providers and knowledge brokers (Howlett 2011, pp. 31-33). 
Other typologies have differentiated the main participants in relation to their location (i.e., core 


174 


Policy Formulation Tools 


actors — professional policy analysts, central agency officials and others) and level of influence (i.e., 
public sector insiders, private sector insiders, and outsiders) (Howlett 2011, p. 33). 

Precisely who formulates policy is ultimately an empirical question. The point we wish to 
make is that it is important to appreciate the variety of actors who might be involved in policy 
formulation activities as they might well have rather different motives and capabilities for using 
particular tools — a matter to which we now turn. 


Policy Formulation Processes and Tasks 


One of the most common ways to comprehend the process of policy formulation is to break it 
down into constituent steps or tasks. Having established the existence of a policy problem (or 
problems) through some form of data collection, the various policy-relevant dimensions of the 
problem are then evaluated to determine their causes and extent, chiefly as a basis for identifying 
potential policy solutions. 

For Wu et al. (2010, p. 40) ‘[u]nderstanding the source of the problem’ is an unavoidable part 
of formulation. They also make the point that rarely is there ‘full agreement over . . . underlying 
causes’ (Wu et al. 2010, p. 40). Like initial problem characterization, evaluation of the causes of 
a problem may thus involve political conflict as different actors seek to apportion blame, reduce 
their perceived complicity or shape subsequent policy responses in line with their interests. 
These characteristics strongly condition the type of tools used. 

Once a broad consensus has been reached on the nature and extent of the problem(s), poli- 
cymakers turn to consider appropriate responses. From the initial information gathering and 
analysis of causes, formulators thus engage in the ‘[s]pecification of objectives’ (Wolman 1981, 
p. 438) or ‘[c]larifying policy objectives’ (Wu et al. 2010, p. 40) stage. Initially, this third step of 
objective specification can involve the determination of the objectives to be met and the tim- 
escales for action (Wu et al. 2010). Again, disagreements over objectives can quickly ensue but 
once they are established, as a fourth step, specific policy options can be assessed, and recom- 
mendations are made on policy design(s). Because any particular problem may have multiple 
potential solutions, each with differing costs and benefits, these options require comparative 
assessment to guide decision-making. 

Prior to the adoption of the final policy, it undergoes a fifth step — design. Having deter- 
mined objectives, various means are available for selection from the toolbox (for example, 
Howlett 2011; Jordan et al. 2012; Jordan et al. 2013b). Determining the preferred policy mix is 
central to design considerations. While typologies also abound in the instruments literature, four 
main categories are evident: regulations, market-based instruments, voluntary approaches and 
informational measures (Jordan et al. 2013b). In addition, the instrument of public spending or 
budgeting may also be identified (see, e.g., Russel and Jordan 2014). Policymakers select from 
these instruments according to a range of considerations that are both internal and external to the 
instrument. This stage of formulation could, according to Wolman (1981, pp. 440-446), con- 
sequently involve the weighing up of several factors: the ‘causal efficacy’ of the policy, ‘political 
feasibility’, ‘technical feasibility’, any ‘secondary consequences’ resulting from the design, instru- 
ment type (regulations or incentives) and the capacity of implementation structures. This cycle 
constitutes the standard steps or tasks of policy formulation. 

As mentioned earlier, all the steps including design may be deeply contested. After all, the final 
architecture of the policy could, once implemented, create winners and losers via processes of 
positive and negative feedback (Jordan and Matt 2014). One means of dissipating distributional 
conflict throughout the entire formulation process is to engage in what Thomas (2001, p. 218) 
terms consensus building or ‘consolidation’, whereby agreement is sought between the various 
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policy formulators and their client groupings. We shall show that a number of formulation 
tools have been developed specifically for this purpose. But while ‘[a]nticipating and address- 
ing the . . . concerns of the various powerful social groups is essential’, consultation may create 
associated transaction costs such as the slowing down of policy adoption (Wu et al. 2010, p. 41). 
A decision can be taken — the subsequent stage of the policy process — once agreement has been 
reached on the chosen course of action. 

Indeed, policy formulation may not culminate in the adoption of a discrete and hence settled 
‘policy’: on the contrary, policies may continue to be (re)formulated throughout their imple- 
mentation as tool-informed learning takes place in relation to their operational effectiveness and 
associated outcomes (Jordan et al. 2013a). As we shall show, many policy analysts responded to 
these discomforting discoveries by offering ever-more-strident recommendations on how policy 
formulation should be conducted (Vining and Weimer 2010; Dunn 2004); notably fewer have 
studied how it is actually practiced (Colebatch and Radin 2006; Noordegraaf 2011). In the fol- 
lowing section, we shall explore what a perspective focusing on formulation tools and venues 
offers by way of greater insight into the steps and the venues of policy formulation. 


The Venues of Policy Formulation 


Policy formulation — like policymaking more generally — occurs in particular venues and can, in 
principle, exist at different levels of governance, as well as within or outside the structures of the 
state. Baumgartner and Jones (1991, p. 1045) have termed these ‘venues of policy action’, going 
on to define them as “institutional locations where authoritative decisions are made concerning a 
given issue’ (Baumgartner and Jones 1993, p. 32). More specifically, Timmermans and Scholten 
(2006, p. 1105) suggest that the venues ‘are locations where policies originate, obtain support, 
and are adopted as binding decisions’. 

Several types of venues have been detected, including within federal, state and local governments 
and within international organizations (Pralle 2003), European Union institutions and national 
governments (Beyers and Kerremans 2012) and various trans-governmental cooperation mechanisms 
(Guiraudon 2002). Venues can include ‘formal political arenas such as legislatures, executives and the 
judiciary, but also the media and the stock market’ and so-called ‘scientific venues such as research 
institutes, think-tanks and expert committees’ (Timmermans and Scholten 2006, p. 1105). 

On this basis, any attempt to categorize the venues of policy formulation should be cognizant 
of the institutional space itself and, significantly, the type of evidence used. With respect to the for- 
mer, when examining formulation, we can more neatly divide venues by functional power rather 
than institutional level or actor group. Here, in terms of relative power, it is national government 
executives that are still arguably dominant globally, despite increasing shifts towards multi-level 
governance (Jordan and Huitema 2014). To give greater analytical purchase to our conceptu- 
alizations, we therefore build on Peters and Barker (1993), Baumgartner and Jones (1993) and 
Timmermans and Scholten (2006) and define policy formulation venues as institutional locations, 
both within and outside governments, in which certain policy formulation tasks are performed 
with the aim of informing the design, content and effects of policymaking activities. 


The Tools of Policy Formulation 


The Turn Away From Policy Formulation Tools and the Turn Back 


Tool-driven or ‘analycentric’ approaches (Schick 1977) initially developed in the fields of defence 

and budgeting, but from the late 1960s, the scope of analytical activities has greatly expanded 

(Parsons 1995; Radin 2013, pp. 17-22; DeLeon 2006) almost as a corollary to the general 
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expansion in the state. As Schick (1977, p. 258) observed: ‘[W]henever positive government 
action has been extended to a new sphere, analytic activity has been sure to follow’. Crucially, 
the increasingly forceful turn towards analycentric tools and methods embedded a linear-rational 
approach to analysis of policy problems in which — to put it simplistically — problems were to be 
identified and then ‘solved’ using analytical tools. 

These tools primarily drew on techniques from operational research and economic analysis, 
including methods for assessing the costs and benefits of different policy alternatives and analysis 
of interacting parts of complex systems. Tools such as cost-benefit analysis (CBA) and computer 
models were to be found in the analycentric ‘backroom’ (Self 1981, p. 222), where political 
‘irrationalities’ could be tempered and policy made more ‘rational’. 

While from the Lasswellian (Lasswell, 1971) perspective, tools were seen as having a central role in 
the development of an integrated approach that united policy researchers with policy practitioners, 
for several reasons, things did not quite match up to this vision. Policy formulation tools were 
gradually marginalized in public policy research, and some fell out of favour with policymakers. For 
example, CBA and integrated forms of planning and budgeting such as the Program Planning and 
Budgeting System (PPBS) fell some way short of initial expectations, and the rise of private sector 
management techniques in running public services (e.g., the New Public Management agenda), 
coupled with the desire to reduce the power and scope of bureaucracy, nurtured a demand for a new 
set of accounting tools for contracting out public services (Mintrom and Williams 2013). 

In time, tool designers and developers became ever more divided into ‘clusters of functional 
interest’ (Schick 1977, p. 260). The idea of an integrated policy analysis for democracy was quietly 
forgotten in the rush to design ever-more-sophisticated tools. Indeed, some have devoted their entire 
careers to this task, only to discover later that relatively few policymakers routinely used the tools they 
had designed (Pearce 1998; Hanley et al. 1990). As Schick (1977, p. 262) had earlier predicted, they 
believed that the route to usefulness was via ever greater precision and rigour — but it wasn’t. 

Nowadays, interest in policy formulation tools appears to be growing strongly once again, for 
several reasons. First, new tasks other than knowledge creation are being found for tools such as 
CBA and indicators, including as a means to implement the New Public Management agenda. 
Second, the emergence of complex policy problems such as climate change has generated a fresh 
wave of interest in more sophisticated policy formulation tools such as scenarios and computer- 
based forms of modeling. Third, the growing interest in policy formulation tools could also 
be seen as one symptom of the gradual rediscovery of policy design as both a policy goal (e.g., 
through state-led policymaking) and a research topic (Howlett et al. 2014). Far from reducing 
the need for state involvement, the emergence of a more complex, networked society and aus- 
terity pressures make it more important for interventions to be carefully targeted and legitimated 
(Howlett and Lejano 2013, p. 12; Xiol Y Ferreira and Howlett, 2021). Finally, the number of 
policy formulation tool types has grown significantly in recent years. And as they have emerged 
from the analycentric ‘backroom’ (Self 1981, p. 222), the expectation has grown that that they 
will respond more sensitively to changing contextual conditions and public expectations. In the 
next section, we attempt to bring a greater sense of analytical order to the expanding list of tools, 
methods, tasks and expectations by taking stock of the literatures to date. 


Formulation Tools: Towards a New Sub-field of Policy Analysis? 


The Literatures on Policy Formulation Tools: Taking Stock 


In attempting to move the study of policy formulation tools back into the mainstream of 

public policy research, we immediately confront a problem — the relative absence of common 

definitions and typologies. Without these, it is difficult to believe that the literature on the topic 
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can be telescoped into a new sub-field. We believe that four literatures provide an especially 
important source of common terms and concepts, which we now briefly summarize. 

The first literature describes the internal characteristics and functions of each tool and/or 
offers tool kits which seek to assist policy formulators in selecting ‘the right tool for the job’. It 
outlines the intrinsic features of each tool, but it does not have a great deal to say about where, 
how, why and by whom (i.e., by which actors and in which venues) they are used and what 
effects they (do not) produce. 

The second is dominated by typologies. Tools can be typologized in a number of different 
ways: for example, by the resources or capacities they require, by the activity they mainly sup- 
port, by the task(s) they perform and by their spatial resolution. Radin (2013, p. 145) opts for a 
more parsimonious framing, distinguishing between two main types: the more economic tools 
such as cost-benefit analysis (CBA) and what she terms the more ‘systematic approaches’, such 
as criteria analysis and political mapping. The problem is that dividing the field in two does not 
really offer much typological variation. 

The third literature adopts a more critical perspective (Wildavsky 1979; Shulock 1999; Self 1981), 
through offering words of caution about expecting too much from tools. It appears to have left a deep 
impression on a number of policy analysts, perhaps sufficient to militate against the development of 
a new sub-field. However, despite these cautionary words, many tools have been developed and are 
very heavily applied in certain venues to routinely produce effects that are not currently understood. 
Hence, questions about precisely where, how, why and by whom they are used remain. 

A fourth and final literature is more strongly focused on the main venues and processes of 
policy formulation rather than the tools. In attempting to understand better and explain how 
policy is made and what influences it, this literature encompasses studies of crucial factors such as 
the utilization of knowledge in policymaking (Radaelli 1995), and the role of power and institu- 
tions. (For an excellent summary, see Sabatier 2005.) The manner in which power and particular 
analytical practices are bound up with one another has been explored in planning/geography 
(see, e.g., Owens and Cowell 2002) and science and technology studies (Stirling 2008). Other 
aspects focus on the political demand for evidence-based policymaking (Sanderson 2002; Shine 
and Bartley 2011). Much of this literature adopts a macro- or a meso-level focus and draws on 
or develops theory. To the extent that it considers policy formulation tools at all, there is a ten- 
dency (although by no means universal) to assume that tools are epiphenomenal and, hence, not 
warranting detailed analysis. Without more detailed research, however, these remain no more 
than untested assumptions. 


Reassembling the Field: A Definition, Typology 
and Analytical Framework 


While keenly aware that typologizing can very easily become an end in itself, we argue that 
developing some kind of workable taxonomy nonetheless remains a crucial next step towards 
enhancing a shared understanding of how policy formulation tools are used in contemporary 
public policymaking. To move forwards, we draw upon Jenkins-Smith (1990, p. 11) by defining 
a policy formulation tool as a technique, scheme, device or operation (including — but not lim- 
ited to — those developed in the fields of economics, mathematics, statistics, computing, opera- 
tions research and systems dynamics) which can be used to collect, condense and make sense of 
different kinds of policy-relevant knowledge to perform some or all of the various interlinked 
tasks of policy formulation. 

We propose that five policy formulation tasks — problem characterization, problem evaluation, 
specification of objectives, policy options assessment and policy design — may be used to structure 
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a typology of policy formulation tools, based on what might be termed the ‘textbook’ character- 
istics of what they may be capable of. For this, we also draw on Dunn’s (2004, pp. 6-7) schema of 
three types of tasks associated with policy formulation tools (problem structuring, forecasting and 
recommending) and de Ridder et al’s (2007) typology of assessment tools. The first two tasks of 
‘problem characterization’ and ‘problem evaluation’ broadly correspond to Dunn’s (2004) problem 
structuring — that is, tools that produce information about what problem to solve. The remaining 
three tasks correspond to Dunn’s forecasting — tools that produce information about the expected 
outcomes of policies — and also recommend tools that produce information about preferred poli- 
cies. Interaction between four key aspects of these tools — together — constitutes our analytical 
framework: actors, venues, capacities and effects. The discussion that follows explores, via this 
analytical framework, how these tasks (or uses) work out in practice. 


Actors 


The growth in policy formulation tools is a tangible manifestation of the broadening and deep- 
ening of the policy analysis and advisory community from one dominated by generalist bureau- 
crats and “econocrats’ (Mintrom and Williams 2013, p. 9) to one comprising a multitude of 
actors within a more open and plural policy advisory system. Instead of ‘speaking truth to 
power’, as Wildavsky (1979) would have it, putting policy formulation tools alongside the actors 
that utilize them provides a sharper picture of how modern policy analysts seek to ‘share the 
truth with many actors of influence’ (Craft and Howlett 2012, p. 85). As a result, the first 
element of our proposed analytical framework concerns the actors who develop and/or pro- 
mote particular policy formulation tools. As discussed generally earlier, three main types of 
actors appear: decision-makers, knowledge producers and/or providers and knowledge brokers 
(Howlett 2011, pp. 31-33). 

Decision-makers at state and international levels have been assiduous promoters of policy 
formulation tools, almost since the dawn of policy analysis (Dunn 2004, p. 40). Nowadays, 
national finance ministries and core executives continue to support the application of indicators 
and CBA through the publication of rules, statutes and best-practice guides under different rhe- 
torical banners, including better regulation, administrative modernization and evidence-based 
policymaking. 

Governmental actors also work within international organizations to share best practices 
on many tools, including scenarios, indicators and CBA. The research arm of the European 
Commission has directly funded many complex computer models (Nilsson et al. 2008) and 
taken active steps to ensure they are more heavily utilized in formalized systems of policy-level 
appraisal. Under the category of knowledge producers and/or providers, there are a myriad 
of actors, in state and non-state settings, who variously invent tools and numerous variants 
thereof (e.g., academics and technical officials in state bureaucracies); refine and update them 
(e.g., scenario developers); and provide the policy-relevant knowledge that is fed into policy 
formulation activities (e.g., statisticians, policy specialists and special advisors). Industry, too, has 
made notable contributions to the development of forecasting, simulation gaming and scenario 
tools. Consultants and think tanks have also created complex modeling tools (such as influential 
energy system models) and scenarios and have also been active disseminators of other tools across 
government. Finally, while knowledge brokers are — in theory — supposed to adopt a more or 
less neutral role between science and policy, in practice, they play an important role in matching 
tools to policy problems (e.g., models to scenarios in processes of integrated assessment). 

Beyond the analysis of actors, adopting a tools perspective on policy formulation — that is, fol- 
lowing a particular tool as it is picked up and deployed in different policy formulation venues — 
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arguably offers a new and potentially fruitful way to ‘open up the black box’ of policy formulation. 
It supplements the standard methods of following issues or focusing on policy advisory systems 
and is discussed further next. 


Venues 


Relatively little is known about how the various tools and venues intersect, either in theory or, as 
importantly, in practice. Thus, the second element of our analytical framework relates to the sugges- 
tion that policymakers apply tools in policy formulation venues, defined on the basis of their loca- 
tion (internal or external to government) and the sources of knowledge that they draw upon (official 
versus unofficial). Although in the past, the standard assumption in policy analysis was that it was the 
state and its constituent organizations that mainly selected and deployed the tools, with a particularly 
strong preference (according to Meltsner (1976) at least) for the more substantive-technical variants, 
such as models and CBA, specific tools do not completely dominate specific venues. 

The purposes to which the tools are put in the various policy formulation venues also exhibit 
a great deal of variation. Purposes can be thought of in at least two distinct senses: vis-a-vis the 
well-known stages or steps of policy formulation and in relation to the pre-existing ‘design 
space’ (Howlett 2011, p. 141): that is, does it seek a radical or a more incremental departure from 
the policy status quo? As regards the former, certain tools appear to be far better suited (and be 
more heavily used in relation) to certain policy formulation tasks than others. The other way to 
consider the purposes to which a tool is put is relate it to the pre-existing ‘design space’. In other 
words, does it seek to implement the existing policy regime (comprising an internally consistent 
set of policy objectives, goals and instruments) (Howlett 2011, p. 142) in a more efficient or 
cost-effective fashion, or does it seek to stretch the existing design space by incorporating new 
problem formulations or radically different policy approaches? In many tool-related literatures, 
this is directly comparable to the distinction between policy analysis that ‘opens up’ debate and 
that which closes it down (Stirling 2008). 

Although the literature on policy formulation tools is still immature, there do appear to be 
clear and discernible patterns in the way that policy formulation tools are used. Whether one 
starts with the tools and looks across to the venues or explores different combinations of tools 
in and across particular venues, the patterns seem to recur and hence, in principle, seem worthy 
of further exploration. Indeed, one especially intriguing possibility is that the most significant 
differentiating factor may eventually be policy type, not venue. 


Capacities 


Third, analysts should examine the relationship between policy capacity and policy formu- 
lation tools. Policy capacity is one of a number of sub-dimensions of state capacity, which 
together include the ability to create and maintain social order and exercise democratic authority 
(Matthews 2012). Broadly, it is the ability that governments have to identify and pursue policy 
goals and achieve certain policy outcomes in a more or less instrumental fashion: that is, ‘to 
marshal the necessary resources to make intelligent collective choices about and set strategic 
directions for the allocation of scarce resources to public ends’ (Painter and Pierre 2005, p. 2). It 
is known to vary between policy systems and even between governance levels in the same policy 
system. Policy instruments and tools have long been assumed to have an important influence on 
policy capacity — if they did not, why use them (Howlett et al. 2014, p. 4)? The fact that they 
are unevenly used over time, for example, could explain why the policy capacity to get things 
done also varies across space and time (Bahr 2010; Wurzel et al. 2013). 
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In principle, therefore, the presence and availability of policy formulation tools help expand 
policy capacities, although we should not automatically assume that the relationship is immediate 
or unidirectional. The fact that critical supporting capacities may not be available in every policy 
system is also important to note. 


Effects 


Finally, what effects, both intended and actual, do the various tools generate when they are 
employed? As we explained earlier, our original expectation was that the tools would produce 
some quite specific epistemic and political effects. But while some evidence is available on their 
wider effects, much more is required. The policy instruments literature has been struggling 
to address this question, at least for implementation tools, ever since Salamon (2002, p. 2) 
speculated that each tool imparts its own distinctive spin or twist on policy dynamics. This 
collective failure probably has much to do with the disciplinary background of the contributors, 
but it also reflects an entirely understandable desire to stay anchored in the relatively clear-cut 
world of textbooks and typologies. Nonetheless, some potentially useful categorizations could 
form the basis of future work. 

For example — and drawing on Turnpenny et al. (2009, p. 648) — a broad distinction can be 
drawn between ‘substantive’ effects (the extent to which tools generate change — or work to 
ensure continuity — in a given policy field) and ‘process-based’ effects (i.e., system-wide effects 
which arise from the use of particular tools). Substantive effects include learning in relation to 
new means to achieve given policy goals (a feature which is predominant amongst the more 
structured procedural tools such as CBA, but also computer modeling tools) through to the 
heuristic-conceptual effects on problem understandings. The procedural effects could be simi- 
larly wide ranging including (re)channelling political attention, opening up new opportunities 
for outsiders to exert influence and uncovering political power relationships. In addition, some 
participatory tools such as the devil’s advocate technique and participatory backcasting have the 
aim of generating new understandings and uncovering extant political power relationships. 

A second important distinction relates to the difference between intended and unintended 
effects. To a large extent, the difference is one of prior expectations, purposes and ultimately val- 
ues. Thus, by their very nature, the more procedurally inflexible tools such as CBA appear more 
prone to performance deficits. But more open, participatory tools can also produce unexpected 
effects. For example, backcasting approaches all too easily entrench political differences and 
forms of participation. Consequently, the new sub-field of policy formulation research should 
be careful to pose more probing questions (e.g., unexpected by whom and why?) rather than 
assume that everything which is unexpected is necessarily bad or otherwise. 

In sum, understanding tool effects arguably constitutes the biggest analytical challenge of all, 
but one which the nascent sub-field of policy formulation is beginning to engage with. A rather 
different lens through which to study the interaction of actors, venues, capacities and effects is 
that of executive oversight and/or political control over non-majoritarian agencies (Turnpenny 
et al. 2009, p. 645). There are, however, many important questions still to be addressed by those 
seeking to move the new sub-field in this theoretical direction. 


Conclusions, New Perspectives and New Challenges 


The existing literatures on policy formulation tools remain fragmented, across not only the main 
tool types but also different disciplines. As policy analysts, it is arguably the one we know the 
least about and is often complex and fluid and usually much less accessible to public scrutiny 


181 


Elisa Helena Xiol Y Ferreira et al. 


than the other stages. For policy analysts, the divide between those tool experts seeking to pur- 
sue research ‘of policy’ and those preferring to undertake analysis ‘for policy’ seems even more 
pronounced than in other comparable sub-areas of policy analysis such as policy instruments. 
Looking through the prism of tools is methodologically advantageous in the sense that, drawing 
on Hood (1983, pp. 115-131), it reduces complexity and permits comparisons to be made more 
easily across time and between different policy areas and political systems. There is potentially 
much more to add to our collective understanding of the tools themselves, which, as repeatedly 
noted, have often been studied in a rather isolated, static and descriptive manner. 

To reinvigorate our understandings, analysts require common concepts, parsimonious defini- 
tions and usable taxonomies. Here, we have sought to supply and then critically reflect on all 
three and now invite readers to apply, test and critique them. Of course, at the level of specific 
tools, debate about definitions, typologies and purposes will undoubtedly continue. We see that 
as a healthy sign yet believe that agreement at the broader level is now needed to generate a 
common and hopefully more fruitful research agenda, perhaps organized around our framework 
of actors, venues, capacities and effects. 

A tools perspective offers insights into governing beyond the formal rules, administra- 
tive systems and constitutions that dominate existing policy analysis. As a result, a renewed 
focus on policy formulation tools can add to our collective understanding not only of policy 
formulation but of public policy more generally. At present, there are no maxims (Howlett 
et al. 2014) of the type found in the policy instruments literature (e.g., escalate slowly up 
the pyramid of intervention) or meta-tools to inform the design of tool packages. Clearly, 
inconsistencies between some tool pairings are more obvious than between others. Multi- 
criteria and participatory approaches do seem to mix more freely with one another than, for 
example, CBA and scenarios. But there is plenty of fresh work to be done on whether and 
indeed why this might be the case. 

Finally, bringing policy formulation tools into the mainstream of policy research may help 
us learn more about ourselves and our multidisciplinary field of policy analysis. To tell the story 
of policy formulation tools is to tell of the emergence and professionalization of policy analysis. 
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INDICATORS AND MEASURES 
AS POLICY TOOLS 


Markku Lehtonen 


Today, indicators are produced and used worldwide: across all levels and sectors of society; by 
public, private and civil society actors; and for a variety of purposes, ranging from knowledge 
provision to administrative control. Indicators are ubiquitous in practically all policy sectors, at 
various stages of the policy cycle. They are typically applied in combination with or in support 
of other policy tools, including evaluation and regulation, but also information, communication, 
popular mobilization and social advocacy, as illustrated, for example, by the ever-expanding range 
of indicators of sustainable development and well-being. The ways of producing indicators range 
from expert-led technical exercises to highly participatory multi-stakeholder processes. Indica- 
tors contribute to characterizing the current situations and framings; conceptualizing problems; 
constructing future scenarios; identifying and shaping policy solutions; and assessing, comparing 
and justifying potential policy options. This chapter elucidates on such diversity of indicators 
and their roles both in formal policymaking and in the numerous informal venues that shape 
policy decisions and their context. Particular attention is given to the various unintended and 
systemic consequences of the production and use of indicators. The chapter ends by identifying 
challenges for indicator work in the era of big data and declining trust in authority. 


Introduction: Indicators as Policy Tools 


Indicators have become a ubiquitous policy tool, produced and used worldwide, across all lev- 
els and sectors of society, by public, private and civil society actors for a variety of purposes. 
Although this chapter deals with them under the category of policy formulation tools, indicators 
are present in practically all policy sectors, at various stages of the classical policy cycle. They 
are typically applied in combination with, or in support of, other policy tools, including evalu- 
ation and regulation, but also information, communication, popular mobilization and social 
advocacy, as illustrated by the ever-expanding range of indicators of sustainable development, 
happiness, and well-being. The ways of producing indicators range from expert-led technical 
exercises to highly participatory multi-stakeholder processes. Such collaborative processes have 
gained increasing popularity as a way of better addressing the diverse user needs and bridging 
the gap between experts and ‘lay’ people. Participatory processes of indicator elaboration can 
be a key source of influence in their own right (Mickwitz and Melanen 2009; Ottaviani et al. 
2021), thanks to the learning effects of interaction and dialogue. Development of information 
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technologies has enabled new methods of engagement via crowdsourcing or interactive uses of 
indicators (e.g., Radermacher 2021). 

A relatively homogeneous community of indicator users and producers — called by some an 
‘indicator industry’ (Sébastien and Bauler 2013) — has emerged as a powerful instrument con- 
stituency promoting the ever-wider application of indicators as a tool for addressing diverse 
societal problems across a wide range of sectors of society. While indicators are produced by 
a wide range of actors, statistical offices play a central role, as gatekeepers and in indicator 
quality control. The lack — or, in their opinion, erroneous ways of — indicator use has become 
a major concern for this indicator community. Often, the debate focuses on the ‘intrinsic’ 
quality of indicators, with the assumption that ‘better’ indicators would lead to more and 
better use and, hence, positive policy impact. The recent debates around a post-truth society, 
the “digital revolution’, and big data have added a new angle to the concerns over indicator 
quality — raising to the fore the question of trust in authorities producing indicators, notably 
the national statistical offices. 

Numerous tensions and ambiguities characterize indicators as a policy tool, and call into 


question the assumption of an unproblematic linear relationship from better indicators through 
greater use to better policy. 

Ambiguities result from the potential of indicators to serve as ‘boundary objects’ (e.g., 
Turnhout 2009; Sébastien et al. 2014; Lehtonen et al. 2016) mediating across different ‘social 
worlds” (Star 2010) and categories. Indicators can thus both shape the prevailing problem 
framings and are shaped by those very framings (Boulanger 2014: 26). Indicators combine quan- 
titative, ‘hard’ facts with ‘soft’ speculation — drawing on quantitative statistical information — yet 
they are necessarily only approximations of underlying phenomena that escape direct measure- 
ment (Turnhout 2009). As a result, indicator development faces the enduring need to balance 
scientific quality and policy relevance. 

Furthermore, indicators can serve centralization of power via control and management 
(e.g., through government performance targets and indicators) or local-level emancipation 
and empowerment (e.g., via participatory indicator processes or application of indicators of 
environmental quality, sustainable development or well-being). The power of indicators is based 
on widespread trust in their quality as an authoritative source of data on ‘reality’, yet the very 
rationale of indicators relies on the idea of mistrustful civic vigilance and citizen control of the 
powers to be. 

Finally, the role of indicators in policymaking can be seen either through the lens 
of ‘informational managerialism’ (Mol 2006: 506), which portrays indicators as a rather 
unproblematic direct input to societal transformations, or through a much more prob- 
lematic framing that highlights the numerous indirect, unintended, unanticipated and 
systemic consequences, including dysfunctional effects such as consolidation of the pre- 
vailing asymmetries of power (Gudmundsson 2003; Hezri and Dovers 2006; Hood 2007; 
Lehtonen 2013; Lehtonen et al. 2016; Lyytimaki et al. 2013; Rydin 2007; Turnhout 2009; 
Radermacher 2020). 

This chapter illustrates this ambiguity of indicators as a policy tool by describing the types 
of indicators (section 2), examining their role at various stages of the policy cycle (section 3), 
surveying the concerns over the scarce use and frequent misuse of indicators (section 4), exploring 
the literature on the indirect consequences of indicators (section 5) and discussing the role of 
indicators in policy instrument mixes (section 6). Section 7 concludes and identifies challenges for 
future indicator work in an era of big data and declining trust in the authority of expertise. 
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What Are Indicators? Definitions, Types, Qualities 
and Production Processes 


Indicators come in various forms, with distinct purposes, functions, disciplinary backgrounds, 
application areas and levels, as well as theoretical and even ideological underpinnings. Two fea- 
tures are common for most definitions of indicators: ‘indication’, implying that an entity that 
is not directly measurable can be ‘assessed using a limited set of measurable parameters’ (Turn- 
hout 2009: 403), and ‘signaling’ — the idea that an indicator needs to be interpreted and given 
meaning (Jackson 2011: 15). As policy tools and instruments in general, indicators not only are 
shaped by dominant framings of social problems but also shape those very framings (Boulanger 
2014: 26). By rendering problems measurable, indicators unavoidably affect the ways in which 
these problems are perceived and addressed. 

Underlying by conceptual frameworks is what determine the criteria and logic for the choice 
of specific indicators, anchor indicator systems in theory and make them comparable with each 
other and communicable to the targeted audience (Gudmundsson 2003; Pintér et al. 2005). 
Often, indicators are associated with specific targets that allow for judgments and assessments 
to be made, enabling indicators to provide early warning through the observation of trends. 
Unlike evaluations, indicators are not always underpinned by a model that establishes cause- 
effect relationships (Gudmundsson 2003). 

Three overlapping yet analytically distinct categories of indicators can be identified: descriptive, 
performance and composite indicators. Descriptive indicators resemble ‘pure data’, which is not 
designed for a specific use and thereby do not, in the first instance, appear as policy tools. Per- 
formance indicators measure the performance of an entity (an organization, sector, city, country, 
etc.) in a given area of activity, judging progress towards a normative goal. Examples range from 
environmental performance indicators (e.g., OECD 1991) to government performance moni- 
toring, or key performance indicators used by an organization or a project. Performance indica- 
tors are frequently applied by government for the purposes of monitoring and accountability but 
can just as well foster learning, policy improvement and demonstration of policy improvement 
(e.g., Boswell et al. 2015). 

Finally, composite indicators (e.g., GDP, ecological footprint, new indicators of well-being) 
seek to present the ‘big picture’ in a manner accessible to diverse audiences, thereby serving in 
particular communication and awareness raising. School and university rankings have become 
increasingly visible and powerful in measuring and signaling the quality of service and thereby 
informing choices, including those that concern resource allocation. Composites can also 
be used for performance benchmarking and accountability and for attributing rewards and 
sanctions (Jackson 2011). Composite indicators of sustainable development are expected to 
provide a common language that enables comparison between theories, engenders collective 
understanding of sustainability, and provides an alternative to GDP as a measure of societal 
progress (e.g., Chancel et al. 2014). 

Composites are not well suited for steering specific policy decisions, given their inability to 
identify causality. However, they can have significant indirect policy impacts as the ubiquity of 
GDP in media and policy debate demonstrates. Ideally, composites inform both the public and 
political debates about objectives and trade-offs, help make explicit the assumptions that under- 
lie policy debates, challenge dominant ways of measurement and help citizens hold politicians 
to account (Seaford 2013). Table 15.1 summarizes the characteristics and expected functions of 
the three broad categories of indicators. 
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Table 15.1 Three Types of Indicators 


Type of Indicator Presentation Functions 
Descriptive Number, grade, time series, ratio or Source of data and information 
dichotomy 
(Often) no policy interpretation 
Performance Comparison to a standard, target value or Measurement and monitoring of 
benchmark goal achievement 
Composite Aggregate index composed of several Snapshot of performance at a given 
variables point in time 
Often combines descriptive and performance Raising awareness, drawing attention 
indicators to a policy issue 


Source: Adapted from Gudmundsson (2003) and Lehtonen (2017) 


Intended Roles of Indicators in a Policy Cycle 


The key concern for indicator producers and a large part of the indicator community is that of 
maximizing the quality of indicators. The attention, therefore, focuses on scientific criteria such 
as reliability, validity, measurability, and representativeness. However, indicator quality crucially 
depends on the intended and actual use of the indicators. The disappointment at the lack of use 
and widespread ‘misuse’ of indicators has given rise to a plethora of guidelines and principles of 
‘good practice’ that seek to better connect indicators with policymaking, enhance trust in indi- 
cators and indicator producers, strengthen the resources and institutional capacities available for 
indicator producers, better match indicators with user needs and abilities, improve communica- 
tion, and include users in indicator design (e.g., Pintér et al. 2005; Seaford 2013; ESAC 2015). 

A look at the intended and actual roles of indicators throughout the policy cycle provides a 
first way of broadening the perspective away from the intrinsic quality of the indicators alone 
towards policy relevance and the potential roles of indicators in the policy processes. 

Although appearing in this book in the section on policy formulation, indicators fulfill mul- 
tiple functions throughout the entire policy cycle. At the agenda-setting stage, indicators can 
help conceptualize problems and characterize the current situations and framings. Indicators 
shape agendas indirectly by framing problems and policies. However, governments can use indi- 
cators intentionally to help shape the policy agenda. For instance, they can use indicators to 
draw attention to specific policy problems, either by publishing indicators or by strategically 
and repeatedly feeding the media with selected indicators (e.g., Marland 2022). Indicators can 
contribute to ‘evaluation systems’ designed for agenda-setting (Pattyn and Bundi 2022). Sustain- 
ability and new well-being indicators — whether composites or sets of indicators — illustrate the 
awareness-raising and mobilization functions, which, in turn, are intimately linked with social 
identities (Hornung and Bandelow 2022). Such indicators are likely to appeal in the first place 
to the ‘green-minded’ groups in society, including cities, communities and companies that wish 
to see themselves as ‘green’, ‘responsible’ and ‘inclusive’ and portray such images to their peer 
communities, clients and decision-makers. 

Indicators are intended to serve several policy formulation tasks, summarized in Table 
15.2). They contribute to structuration and conceptualization of policy problems whereby 
policymakers select the forms of relevant evidence and types of action, or when issues 
themselves produce specific types of evidence amenable to description via indicators (Turn- 
penny et al. 2015: 8). 
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Indicators can (1) provide baseline information on policy problems (e.g., state of the environ- 
ment or sectoral indicators), (2) offer evidence on causal relations and scale of problems, and (3) 
help articulate values through participatory indicator elaboration or as inputs to participatory 
processes of policy formulation. By characterizing and framing problems, indicators fundamen- 
tally shape policy formulation while at the same time being shaped by issue framings. The role 
of GDP is again illustrative. GDP reflects societal trends, having become the overwhelmingly 
dominant indicator of progress as economic efficiency grew into the overarching policy objec- 
tive in the post-war era. On the other hand, GDP frames the question of societal progress in 
ways that may appear as self-evidently ‘correct’ for economists but can be problematic from a 
societal perspective (e.g., Morse and Bell 2011; Boulanger 2014). 

Indicators help specify objectives. In catering to technocratic and deliberative ideals — 
seemingly exact, rigorous, scientific and objective information on the one hand and policy- 
relevant, tailor-made and partly subjective evidence on the other — indicators can operate as 
boundary objects that shape the conceptualization of problems and mediate between various 
groups and even ‘social worlds’, allowing these to collaborate in the absence of full consensus 
on the meaning and functions of the indicators in question (e.g., Turnhout 2009; Star 2010; 
Lehtonen et al. 2016). 

Forward-looking indicators used as part of scenario exercises do this by producing informa- 
tion about the expected outcomes of policies or helping envision alternative futures. Scenario 
and visioning processes can indeed partly address a frequent criticism of indicators, namely, their 
alleged lack of timeliness and inability to examine future trends (e.g., Lehtonen 2017; Sébastien 
et al. 2014). Especially composite indicators also allow various groups to disseminate and pro- 
mote their preferred visions and worldviews (Sébastien and Bauler 2013; Sébastien et al. 2014). 
By quantifying and simplifying, indicators also shape and structure formal and informal models 
(Seaford 2013) and scenarios, and render problems more manageable. 

Indicators enable the assessment of policy options via comparison of the possible impacts 
of diverse policy options and the exploration of past and future trends. Comparisons can 
draw on historical data and experience or on modeling and anticipation of the future 
impacts of policy options. Indicators are routinely used as part of such tools as cost-benefit 
and cost-effectiveness analysis, multi-criteria analysis, risk assessment, time-series and sta- 
tistical analyses, expert judgment (e.g., Delphi), forecasting, and multi-agent simulation 
(Lehtonen 2015, 2017). 

Indicators play an auxiliary role in the identification and design of policy options, that 
is, in the production of policy recommendations, based on the evaluation of the likely 
effectiveness of policy instruments or mixes. At this stage, indicators help define what is 
doable — identify the relevant and realistically viable policy options. By quantifying, measur- 
ing, aggregating, and prioritizing information, indicators delimit the range of policy options 
considered relevant: indicators shape perceptions of which policies offer feasible and appro- 
priate means of promoting the chosen aims and objectives. For instance, climate policy 
choices depend partly on whether one takes the total greenhouse gas emissions or only CO2 
emissions as the leading indicator. 

In addition to policy formulation, indicators support decision-making tools such as scenario 
planning, multi-criteria decision-analysis, forecasting, and backcasting (Alves Furtado 2022; 
Marttunen et al. 2022; van der Steen 2022). As part of communication and policy-steering 
efforts by governments, indicators contribute to policy implementation (Howlett 2022). Sup- 
porting assessment, evaluation, and monitoring, indicators serve as a tool for justifying and 
legitimizing policies, ex ante or ex post. The functions of policy tools hence overlap and interact 
with each other. 
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Table 15.2 Summary of Indicators as Policy Formulation Tools 


Policy Formulation Task Contribution of Indicators 
Problem structuring and Baseline information (state of the environment indicators, sectoral 
conceptualization indicators, etc.) 


Participatory elaboration of indicators 
Indicators as input to participatory policymaking 
Indicators framing policy problems 
Indicators reflecting dominant framings 
Specification of objectives Forward-looking indicators as inputs to scenarios and visions 
Quantification and simplification 
Translation of broad policy aims into specific goals 
Indicators as carriers of visions and worldviews 
Assessment of policy options Indicators as input to formal assessment methods, designed 
to compare impacts from policy options (e.g., CBA, cost- 
effectiveness analysis, multi-criteria analysis, risk assessment, 
time-series analyses, statistical methods, Delphi technique, 
economic forecasting, multi-agent simulation) 
Identification and design of policy Indicators as input to formal assessment methods, to define what 
options and recommendations is doable 


Source: Adapted from Lehtonen (2017) 


Intended Use and Actual Misuse of Indicators 


A second way of broadening out of the perspective from the mere intrinsic quality of the indica- 
tors focuses on the potential users, their needs and the complex behavioral aspects that govern 
the reactions of target groups to diverse policy measures (Capano and Howlett 2020). However, 
the users often, if not mostly, use the tools in surprising ways that often sharply contrast with 
those envisaged by their developers. That user needs constitute a major consideration should 
seem obvious, given that indicators are produced for a specific purpose and use in mind. Yet 
academic debate, in journals such as Ecological Indicators and Social Indicators Research, typically still 
focuses on the search for scientifically more robust indicators, whereas policy debates in interna- 
tional arenas, such as the OECD and Eurostat, tend to give greater emphasis to policy relevance. 

Two key concerns characterize debates within the indicator community relating to indicator 
use. First, there is a general feeling that policymakers seldom use indicators, and second, even when 
they do, indicator designers tend to see this use as selective and incorrect. The alleged ‘misuse’ is 
generally qualified either as intentional ‘cherry-picking’ of data or unintended misuse following 
from ignorance or lack of ‘statistical literacy’, for example, when indicators are interpreted out of 
context, used as political ammunition or simply ignored. Indicator designers frequently condemn 
such ‘misuse’ but consider its reasons as being beyond their control. However, as Capano and 
Howlett (2020) point out, anticipation of the behavior of policy target groups should constitute an 
integral component of policy formulation and tool design, rather than being seen as a mere issue 
of implementation or an external factor that lies beyond the remit of indicator designers. 


Statistical Literacy as an Antidote to Fake News? 


In addition to more collaborative indicator work, statistical literacy is increasingly evoked as a way of 
enhancing use and as an antidote to the ills of the alleged ‘post-truth’ era of populism, polarization of 
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societal debate, and mistrust of authority. Within the indicator community, the loss of public trust in 
official statistics is a major concern (e.g., Davies 2017; Radermacher 2020, 2021). While the decline 
of trust in statistical authorities is by no means equally distributed across countries (Lehtonen 2019), 
there are fears that loss of trust and concerns for potential fragmentation of statistical information, and 
exponential growth in access to diverse data will undermine democracy and lead to the disappearance 
of a common language for public discourse (Davies 2017; Radermacher 2021). 

The answer would lie in investment in education, designed to improve the understanding, skills, 
and abilities of citizens to handle facts, graphics, maps and indicators (Radermacher 2021). This 
way, the population and the ‘political scene’ would better understand the differences in the qual- 
ity of statistical information, manage to distinguish between fake and fact, and gain confidence in 
statistical authorities (ibid.). Trust would here appear as ‘the main and overarching goal of statisti- 
cal governance’ (Radermacher 2020: 140). Based on the assumption that trust increases alongside 
experience, a major challenge for statistical offices would then be to turn today’s potential users into 
real users, who would become more trustful thanks to first-hand experience of indicators. Further 
trust-building measures would include international processes of monitoring and quality control to 
ensure the independence of statistics, institutions, certification systems for indicators produced by 
actors other than the statistical offices, and communication on the democratic and public-service 
values of statistics and the associated institutions (e.g., Radermacher 2021). 


Indicators as a Tool for Civic Vigilance 


Trust in indicators is certainly vital, yet blind trust in indicators as a presumably incontestable and 
authoritative reflection of truth can be just as damaging as generalized mistrust. The very identity of 
statisticians rests on their role as custodians of objective, incontestable data in defense of democracy 
and the underprivileged. However, trust in either the benevolence of the state bureaucracy or in the 
willingness and ability of individual indicator experts to place ‘the public good’ above their particular 
interests should not be taken for granted (Rafols, p. 9). For example, the strong social ties within 
‘indicator communities’ can undermine the general interest at the cost of particular interests. These 
communities of indicator producers and users — e.g., the statistical offices and their main clients — 
usually share similar mental frameworks, epistemic beliefs, and policy objectives; hold strong trust in 
official statistics; and have experience-based reasons to do so. The blurring of boundaries between 
the users and producers of indicators highlights a point made by Capano and Howlett (2020): the 
choice of policy tools by policymakers is never a fully rational process driven by the search for the 
best tool for the purpose. Instead, it is highly influenced by policymakers’ values, knowledge, and 
experience — including experience of collaboration and networking. In such a context, the vast 
majority of ‘citizen non-users’ may indeed have good reasons for mistrust. 

Obviously, skepticism and mistrust are warranted in relation to the assumption that the policy 
goals manifested in an indicator indeed lead to outcomes that are positive for the public interest. 
Rather than unconditional trust in statistical offices and indicators, what is needed is conditional 
trust and ‘prudent’ skepticism, in the spirit of ‘trust but verify’ (Norris, forthcoming). Indicator 
work can, indeed, provide an arena for this kind of mistrustful civic vigilance in the service of 
liberal democracy (Lehtonen 2019). 


Beyond Use: The Unintended and Unanticipated 
Consequences of Indicators 


On the surface, the debates between experts stressing the scientific quality attributes of indica- 
tors and others focusing on policy relevance seem to juxtapose two contrasting views. However, 


191 


Markku Lehtonen 


both tend to reduce the question to indicator quality, the only difference being whether quality 
is judged in terms of internal scientific criteria or the presumed needs of policymakers. The 
third way of broadening out the perspectives, therefore, goes beyond the questions of active 
use of indicators and embraces instead the numerous unintended, unanticipated and systemic 
impacts from indicators and indicator elaboration processes. 

Greater indicator use does not automatically mean ‘better policy’ (e.g., Sébastien et al. 2014; 
Lehtonen et al. 2016; Lehtonen 2017). Highly used indicators are not always highly influential, 
whereas those that are not actively used may generate significant indirect impacts, either positive 
or negative (Lehtonen 2013; Lyytimaki et al. 2013; Sébastien et al. 2014). It is here that the idea 
of policy tool mixes shows its full value: it is hardly relevant or even possible to analyze the use and 
influence of indicators in isolation from the context of multiple policies and tools within which the 
indicators are produced and applied. Indicator producers may regret the lack of use, yet the unin- 
tended, unanticipated and systemic impacts of indicators are powerful, regardless of the intended 
and expected use. Rather than acting directly on the policy target, indicators can operate as ‘triggers’ 
that activate sequences of individual and collective behaviors, producing specific outcomes, often 
through complex pathways and with a considerable time lag (cf. Capano and Howlett 2022). Such 
effects can be synergistic, counterproductive, or additive in relation to those resulting from other 
policy tools (Maor and Howlett 2022), and they can stem just as well from the indicator produc- 
tion processes as from the use of indicators. These impacts can affect the operation of administrative 
structures and societal institutions; shared norms and understandings; and the legitimacy of policies, 
decisions and actors, as well as established ways of policymaking (Hezri and Dovers 2006; Zittoun 
2006). They can indirectly foster social learning, such as networking, and greater focus and motiva- 
tion among policy actors, or “political learning’, whereby policy actors learn to better drive their 
own interest, possibly against the general interest (Hezri and Dovers 2006). The unanticipated uses 
and consequences, in turn, feed back on indicators. Moreover, use and production are interrelated: 
a process of indicator production seen as legitimate is likely to foster greater use. 

Indicators carry with them diverse ideologies, policy paradigms and belief systems (Capano 
and Howlett 2020). The different indicator systems and communities within the ‘indicator 
industry’ bring with them distinct values and ideologies, including contrasting views on quanti- 
fication. For instance, performance indicators reflect an ‘audit culture’ (Hicks et al. 2015) quite 
distinct from the participatory ethos of many of the well-being and sustainable development 
indicator systems. Indicators frame problems and solutions, disseminate and promote world- 
views and visions, and shape policy priorities and performance criteria. They are ‘performative’, 
that is, the conceptualizations that they produce not only describe the world but also act upon 
it (e.g., Waterton 2002; Mickwitz and Melanen 2009). Obviously, indicators also provide useful 
‘ammunition’ for participants in political debates (Sébastien et al. 2014). Above all, indicators 
render problems manageable through quantification. While relatively unproblematic in policy 
areas and issues that, by their nature, lend themselves to quantification (e.g., pollutant emissions, 
wealth distribution, or unemployment rate), indicator use can collide with the very policy cul- 
ture in many sectors or on certain policy issues. 

A rich and diverse body of critical scholarship has highlighted the potentially perverse and 
socially regressive outcomes of indicators. These include critique of indicator work for its exces- 
sive ‘trust in numbers’ and technical rationality — belief in the ability of numbers to turn the 
incommensurable commensurable. Indicators would have a tendency to depoliticize issues by 
reducing normative choices to presumably neutral, incontestable and commonly agreed-upon 
figures, thereby concealing the political choices embedded in the seemingly technical processes 
of indicator design (Ogien 2013; Sébastien and Bauler 2013). Rafols (2019) evokes the ‘fallacy 
of misplaced concreteness’ and false precision as common characteristics of indicator work. 
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Ogien (2013) regrets statistics authorities’ efforts to ‘clean up’ indicators of politics, to cher- 
ish the false image of objective and value-free indicators. These efforts include maintaining the 
independence of statistical offices in relation to the government, but they often also reflect aver- 
sion towards innovative indicators, such as composite indicators of sustainable development and 
well-being, seen by statistics authorities as excessively subjective and value laden (Sébastien and 
Bauler 2013; Lehtonen 2017). Scholars have pointed to the potentially perverse systemic impacts 
that follow from the intimate relationship between indicator systems and the broader dynamics 
and trends in policymaking (Rydin 2007). Concepts used to capture the systemic impacts include 
governmentality (Rydin 2007), government and management by numbers (Hood 2007; Jackson 
2011; Lascoumes and Le Galés 2005), boundary objects (Sébastien et al. 2014; Star 2010), infor- 
mational governance (Lehtonen et al. 2016) and instrumentation (Zittoun 2006; Lascoumes and 
Le Galés 2005). Indicators are seen to incite behaviors aligned with market society and foster both 
inclusion and exclusion via simplification (making problems accessible to non-experts while at 
the same time legitimizing experts as the only true ‘masters’ of numbers) (Zittoun 2006). Even 
the potential of participatory indicator exercises is limited by their embeddedness in the broader 
dynamics and trends in policymaking (Rydin 2007), such as evidence-based policy. 

Performance measurement indicators have faced particularly harsh criticism. Ideally, they 
can provide greater accountability, efficiency and citizen control over policymakers. However, 
even when used with caution and in full awareness of their limitations, performance indicators 
tend to be complex, opaque and reductionist. Rather than enabling dialogue and deliberation, 
especially when coupled with sanctions, they can erode the very foundations of the system they 
are to support by encouraging goal shifting, ‘gaming’, and dissimulation and distortion of data; 
reproducing prevailing power structures; and undermining trust, responsibility, innovation, cre- 
ativity, ambition, and plurality of values and points of view (Hood 2007; Jackson 2011; Perrin 
1998; Ogien 2013). They may instill a management rhetoric to sectors with a ‘non-managerial’ 
tradition (Smith 1990, 1995; Pidd 2005; Bevan and Hood 2006) or ‘crowd out’ from policy 
agenda topics difficult to capture through indicators (Boswell et al. 2015). This perspective 
opens up the possibility that in some contexts, manipulative use can engender socially desirable 
outcomes, and ‘correct’ may instead give rise to perverse effects. The positive upshot is that, 
although frustrating for indicator communities, the lack of use and impact may sometimes be a 
blessing for society (Boswell et al. 2015). 


Discussion: Indicators as Part of Instrument Mixes, 
Bundles and Portfolios 


As policy tools in general, indicators are seldom, if ever, used alone as an isolated policy tool. 
Rather, indicators form part of mixes, bundles and portfolios of a range of policy tools (Capano 
and Howlett 2020). Ideally, indicators then act in synergy, complementing and supplementing 
the tools they interact with. For example, environmental performance indicators and environ- 
mental performance evaluation are more effective together than when each is applied sepa- 
rately. Indicators can support policy strategy work — by providing essential information and 
legitimacy — or operate in conjunction with economic or command-and-control measures 
designed to induce behavior change. Indicators are routinely part of diverse collaborative poli- 
cymaking efforts (Siddiki et al. 2022), scenario construction, or task forces and commissions, 
thus facilitating policy formulation. 

While maximizing synergies and complementarities is the logical primary objective (Capano 
and Howlett 2020), itis undermined by at least two factors. First, indicators can produce effects that 
counteract the objectives of other policy tools. This can be an occasional and context-dependent 
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event or, alternatively, reflect a conflict between the very culture and philosophy of indicators 
and the policy area or policy tool in question. The growing literature on the role of indicators as 
a governance tool has highlighted the numerous unanticipated and indirect impacts of indicator 
systems, notably those associated with the use of quantitative indicators to measure phenomena 
of essentially qualitative nature. Second, because the policy objectives and objectives of indicator 
use are not necessarily shared by all involved parties, it is not necessarily true that greater indicator 
impact and synergy between indicators would produce socially desirable outcomes. Yet as long 
as the objectives, targets, indicators and underlying values have been democratically debated and 
agreed on, coherence across tool mixes can be seen as a reasonable aim. 

The ‘layering’ of policy tools — that is, path dependence that leads to unintentional and 
unplanned policy tool mixes — may not be harmful when it fosters a fuller expression of and 
dialogue between a plurality of views, values and perspectives in society. The creation of instru- 
ment constituencies that drive policy — another characteristic of policy tools — is potentially 
more problematic. In promoting their own favorite indicators and indicator sets, the likeminded 
and closely knit communities of indicator users and producers may propagate values, cultures, 
and policy tools and practices that may be suboptimal for tackling the problems at hand and may 
lack democratic backing. 


Current and Future Challenges of Indicator Use 


Developments in information technologies, including big data and artificial intelligence, repre- 
sent not only a technological but also an institutional and policy challenge. In an era of faltering 
trust in authority and expertise, statistical authorities will have to innovate, notably by engag- 
ing with data users and new data communication and visualization techniques. To build trust, 
authorities and governments might do well in actively nurturing mistrustful ‘civic vigilance’. 
This could include supporting civil society organizations that challenge official indicator work, 
or actively feeding debate on the framings and values that underpin distinct indicator systems. 

Three challenges for policy tool research can be highlighted. First, this research could use- 
fully explore ways of making trust and mistrust operate in tandem, that is, exploring means of 
combining the trust- and mistrust-based functions of indicators within instrument mixes that 
maximize synergies and minimize the counterproductive effects. 

Second, a shared task for authorities and scholars would be that of elucidating the relation- 
ships between data, statistics and indicators on one hand and policy on the other, in the new 
context of ‘big’ and open data. The statistics community has taken steps in this direction (e.g., 
Radermacher 2021), yet the work remains incipient. 

Third, little is known about the variation between countries and sectors in terms of the role 
that indicators play as policy tools. Given the centrality of unintended and unanticipated sys- 
temic effects of indicator work, cross-cultural comparison of such effects appears as particularly 
timely and valuable. 
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A TOOLS-BASED APPROACH 
TO UNDERSTANDING AND 
OPTIMIZING POLICY ADVICE 
IN POLICY MAKING 


Jonathan Craft 


Policy advice is a central preoccupation for a variety of actors located in and around 
government engaged in policy formulation. Public servants are tasked with generating it, we 
hope policymakers are asking for and receiving quality supplies of it to help inform their 
decisions, and several ‘outsiders’ are in the business of providing policy advice whether solicited 
or not. This chapter takes a tools-based approach to exploring the substantive, procedural, 
and temporal aspects of policy advice in relation to how governments generate and evaluate 
policy options and make decisions. Additional attention is paid to the partisan-political and 
nonpartisan aspects of how policy advice, as a tool, impacts the substance of policy matters 
but also the processes by which policy is developed and used by policymakers. The chapter 
argues that the emphasis on procedural information—based public information campaigns can 
be broadened by recognizing that policy advice, as a tool, has inward- and outward-facing roles 
at various stages of policymaking. Government advisory activity can and does move markets 
and impact citizen and firm behavior and has major implications for the supply and demand 
of internal policy work of governments. A series of illustrative examples are used demonstrate 
the breadth and diversity of policy advice, as a policy tool, and the application of a tools-based 
approach to improve how we think about systems of advice and the role of decision-makers in 
managing and optimizing those systems. 


Introduction 


Policy advice is a central preoccupation for a variety of actors located in and around government 
engaged in policy formulation. Public servants are tasked with generating it, and we hope 
policymakers are receiving (and using) quality supplies while various ‘outsiders’ provide it, 
whether solicited or not. The art and craft of giving and using advice have always been 
challenging but are seemingly further complicated by an increasingly contestable advisory 
environment, the complex nature of many policy problems, and political and media cycles 
that reward expedience over thoughtfulness and longer-term consideration. Policy issues and 
governments are often awash in policy advice, but its quality and useability remain matters of 
ongoing concern and debate. 
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This chapter contributes to the collection’s focus on policy tools by applying tools-based 
research to policy advice. This is novel in that few studies of policy advice have explicitly consid- 
ered how, as a policy tool, advice is used and impacts policymaking. The tools approach makes 
sense given the core role of advice in policymaking but also given the number and types of 
advisory tools, including written briefing/decision notes, cabinet submissions, verbal briefings, 
public consultations, advisory commissions and boards, and white and green papers (Prasser 
2006; Craft and Halligan 2020; Pierre 1998). 

Next, recent developments in policy advice research are reviewed to establish a common 
footing and vocabulary. Then, the well-known Hood (1986) instrument classification approach 
that arrays tools based on their government resource use is applied with a focus on “information- 
based’ policy tools. The approach argues that government’s ‘nodal’ or central position within 
broader networks confers it with privileged abilities to collect and disseminate information that 
can be used to affect behavior and achieve policy outcomes (Hood 1986, 2007; Bemelmans- 
Videc et al. 1998; Howlett 2009). 

To date, studies of information-based policy instruments have focused predominantly on the 
dissemination or communications and public information campaigns undertaken by the state. 
This chapter argues that this is unnecessarily narrow. Instead, as an information-based policy 
tool, policy advice is argued to play an important function throughout the policy process within 
and outside governments. Examples are used to draw out the procedural and substantive nature 
of policy advisory tools as well as their applicability to the ‘front end’ and ‘back end’ of the 
policy-making process. 

The chapter closes by drawing out the clear parallels between instrument and policy advice 
research, which have both increasingly sought to grapple with the fact that neither operates in 
isolation. Instead, both are now understood to operate alongside other instruments/many forms 
of advice in so-called instrument mixes or policy advisory systems (PAS). The chapter concludes 
by pointing to the opportunities and the current imperatives for thinking about and studying 
policy advice as a policy tool. 


Policy Advice and Tools: Common Ground and 
Paths Forward in Policy Making 


In a similar way to that by which the policy tools literature evolved from a set of descriptive 
taxonomies to considerations of how and why tools are selected, deployed, and mixed, what 
constitutes policy advice and how its generated, brokered, and consumed have evolved in prac- 
tice and as subjects of study. Early on, policy advice was often thought of broadly in terms of 
government knowledge utilization or, more narrowly, as part of the policy-making process 
(Peters and Barker 1993; Scott and Baehler 2010). For the insider, policy advice could simply be 
an ‘output’ (DPMC 2014), which at its core could be seen as ‘covering analysis of problems and 
the proposing of solutions’ (Halligan 1995, p. 139). 

In this way, policy advice was often relegated to the formulation stage of policymaking. 
However, as we will see later in this chapter, this view has changed, given evidence of a broader 
set of practices and activities related to advisory work that span the policy process and often 
involve a range of actors beyond those of the professional public service (Vesely 2017; Craft 
2015; Scott and Baehler 2010). 

Policy advice is now understood to be linked to front-end policy making activities like those 
associated with the construction of policy problems and objectives, formalization of policy agen- 
das, and policy formulation (Craft and Halligan 2020; Bahler and Scott 2010). The ‘back end’ 
of policymaking or implementation and evaluation activities have also always featured policy 
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advisory work with governments seeking to understand how to work with street-level bureau- 
crats, communities of practice, other orders of government, and a range of actors to operational- 
ize policy and to learn about and evaluate its effectiveness (Howlett 2012). 

The renewed attention of governments to questions of ‘what works’ along with a focus on 
‘delivery’ and managing policy performance have seen back-end information tools like advice 
via performance dashboards, ‘stock takes’ involving the appraisal of how policies are faring, and 
audits and evaluations become common policy instruments in modern governance (Halligan 
2020; Diamond 2020; Lindquist 2006; Gregory and Lonti 2008; Gold 2017). Boston (1994, 
p. 3) offers a useful approach to help make sense of the complex nature of policy advice by 
distinguishing between two general high-level categories of advice: ‘strategic’ and ‘operational’. 
As he puts it: 


‘[S]trategic’ policy advice involving the production of well-researched, in-depth 
reports for political executives on various matters of public policy (e.g. concerning the 
role of the state as funder, provider, regulator), setting out the issues and exploring the 
options; and ‘operational’ policy advice which is concerned with the issues of imple- 
mentation and the administration of government programs (e.g. advice on drafting or 
amending laws and regulations, technical advice of various kinds, advice on appoint- 
ments to public organizations, the monitoring of policy outcomes). 


These distinctions draw out that there are a range of purposes to giving policy advice, which 
are connected to the fundamental starting points of policymaking in constructing and formal- 
izing policy problems and generating options to respond to them, but also to operational mat- 
ters involving how to do it. This chapter uses these categories to examine how policy advisory 
instruments can be conceived of in relation to the arc of policymaking from ‘front-end’ to 
‘back-end’ policy work. These categories can also subsume further distinctions related to the 
partisan-political advice being provided to officials via their politically appointed staffs or oth- 
ers, which complements the non-partisan advice provided in most public service systems (Craft 
2015; Craft and Halligan 2020). 

Advisory research increasingly features a consensus that policy advice now involves a range of 
‘hard’ and ‘soft’ competencies and practices. These include rigorous formal professional policy 
advising — based on the standardized practices and formal techniques of policy analysis such 
as cost benefit and the like — but also research, proposal development, consultation, managing 
political processes, and evaluating outcomes (Gregory and Lonti 2008, p. 838). The ‘soft craft’ 
of advisory work involves marshalling tacit and experiential knowledge, consultation, brokerage 
and negotiation, advocacy, and political acuity, among others (Bromell 2017; Tiernan 2015; 
Prince 2018). Current views understand policy advice as including the provision of recommen- 
dations and guidance and the articulation of preferences in support of policy work (Craft and 
Halligan 2020). That is, not simply generating inputs for formulation and decision-making but 
again more broadly applicable as a range of activities essential to policy processes. 

A second pertinent feature of policy advice research, which shares much with tools-based 
approaches, has been the focus on the substantive and procedural nature of those activities: for 
instance, with major debates and research programs looking at substantive aspects of advice 
related to the content of policy advice. This includes long-standing debates about the nature 
and uses of evidence-based policy advice versus partisan-political forms of advice and distinc- 
tions based on the policy domain and nature of various policy problems (Head 2008; Craft 
2015; Craft and Wilder 2017). Procedurally, researchers have noted the closed or open nature 
of advising processes, the role of various actors and rules that structure advisory process, and the 
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technocratic and democratic implications of policy advisory processes, as well as the procedural 
implications of short- versus long-term forms of policymaking (Craft and Howlett 2013; Pierre 
1998; Prasser 2006). 

A final development of note has been the attempt to make sense of the growing pluralism and 
complexity of advising by thinking about the systems of advice that exist in any given jurisdiction. 
The policy advisory system (PAS), or the assemblage of units and practices that exist at a given time 
that governments and other actors engage with for policy purposes, has helped broaden the focus 
of analysis from overreliance on questions of public service capacity and advisory performance to 
a synergistic frame of analysis that attends to the configuration, operation, effectiveness, and evolu- 
tion of the systems themselves (Halligan 1995; Craft and Wilder, 2017). Readers will think about 
their own countries or cities, where they can readily identify public service staffs but also political 
parties, think tanks or private sector consultancies, academics, or the various international non- 
governmental agencies like the United Nations, International Monetary Fund, or World Bank 
that may be active on policy issues in their jurisdictions. PAS is helpful for recognizing that several 
policy advisory components exist (e.g., types of policy advisors, advice, and advisory practices), 
and important distinctions can characterize their respective configurations and operation in various 
jurisdictions and domains. Despite these advances, PAS falls short in recognizing and accounting for 
how, as a policy tool, advice can be wielded by governments seeking to achieve policy outcomes. 


Policy Advice as an Information-Based Policy Tool: 
Old and New Paths of Study 


Policy tools, or the techniques through which governments generate, evaluate, and imple- 
ment policy options, are numerous, and this collection provides a range of different typologies 
and approaches. Using Hood’s (1986, see Table 16.1) well-known typology that arrays policy 
tools by primary governing resource used, we see that policy advice is a ‘nodality/information’ 
policy tool. Hood (2007, p. 129) explains that nodality-based tools relate to the ‘capacity of 
government to operate as a node in information networks — a central point of contact’. Thus, with 
respect to information tools, governments are nodal in the sense that they provide, collect, block, 
and broker information resources. A variety of information-based techniques are available to 
governments, ranging from general information campaigns and information or data collection 


Table 16.1 Examples of Policy Instruments, Classified by Principal Governing Resources Used 


Nodality/ 
Information 


Authority Tieasure Organization 


Information Command-and- Grants and loans Direct provision of goods 


collection and control regulation and services and public 


release enterprises 
Advice and Self-regulation User charges Use of family, community, 
exhortation and voluntary 
organizations 
Advertising Standard setting and Taxes and tax Market creation 
delegated regulation expenditures 
Commissions Advisory committees Interest group creation Government reorganization 


and Inquiries 


and consultations 


Source: Adapted from Hood (1986) 


and funding 
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to government advertising; commissions of inquiry, access to information, and privacy acts 
all fall within the rubric of information-based policy instruments. Advice was included as the 
exemplar of this in Hood’s (1986, 2007) approach to classifying and understanding policy tools 
but has not been the subject of significant tools-based analysis. 

Many of these will be familiar to readers who may have seen their government mount 
public information campaigns such as those designed to promote recruitment to the armed 
services or inform voters of election dates along with what is required to vote. Others may 
recall commissions of inquiry or bespoke advisory committees being struck to deal with major 
government scandals or public health/safety emergencies like COVID-19. Some may even have 
tried to request information directly from government only to have been denied or receiving 
heavily redacted government materials. Information-based policy tools, as Vedung et al. (1998, 
p. 103) put it, ‘[Cover] government-directed attempts at influencing people through transfer 
of knowledge communication of reasoned argument, and moral suasion in order to achieve a 
policy result’. It includes the ability of government to disseminate information through public 
information campaigns but, as this chapter argues, also through their advisory activity, given 
governments nodal powers of impacting how information is generated, brokered, and consumed 
within and around government in the policy process. 

Within the tools literature, some work has been done on exploring the use and effectiveness 
of information-based policy tools. To date, most of the academic study of information-based 
policy tools has focused on the outward communication function of government, typically 
focusing on the use and effects of public information campaigns or their ability to structure and 
limit the processes by which information is collected or released via secrecy or administrative 
discretion (Adler and Pittle 1984; Bennett and Raab 2003; Salmon 1989; Stanbury and Fulton 
1984; Weiss and Tschirhart 1994; Howlett 2009; Howlett et al. 2010a, 2010b). 

A key argument of this chapter is that this is an overly narrow view. Firstly, on the grounds that 
governments often provide policy advice that is public or outwardly directed to citizens, business, 
or international actors or jurisdictions that is not always part of an organized ‘campaign’. A good 
example would be the release of public health policy advice regarding COVID-19. Governments 
around the world and their respective public health agencies and non-governmental agencies 
like the World Health Organization (WHO) all engaged in policy advisory activity designed to 
affect the behavior of states, citizens, and domestic and international firms and organizations. 
While some policy advice was coordinated and campaign-like, much of it was not. It included 
the released public advice about the fundamental understandings of what the COVID-19 disease 
was, its origins, and its transmissibility (e.g., via droplets versus aerosol transmission), along with a 
range of public health advice regarding the need for and efficacy of physical distancing, masking, 
and even the lockdowns of entire communities, while others published advice on corresponding 
economic and social implications or countermeasures (Boin et al. 2021). 

Second, policy advice not only satisfies the traditionally dominant ‘outward’ dissemination 
view of information-based tools but also points to a sizable and diverse array of ‘internal’ uses 
and instrument types used in the policy process. Policy advice fundamentally meets the spirit and 
intent of information tools as ‘government-direct government-directed attempts at influencing 
people through transfer of knowledge communication of reasoned argument, and moral suasion 
in order to achieve a policy result’ (Vedung et al. 1998, p. 103). 

What is distinct is the target group, which is internal and diverse, including a broad range and type 
of public sector staffs and elected officials, who are in effect trying to influence other public service or 
government actors. Public servants’ raison d’étre in large part involves advisory work to inform policy 
deliberations and how best to implement democratic decisions in operational contexts by providing 
their best professional free and frank advice (Scott and Baehler 2010; Craft and Hallgan 2020). 
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Policy advice is one of the forms of oxygen that facilitates the operation of government at 
a basic level. Public servants and departments are always working on policy issues and generat- 
ing policy advice: penning briefing notes or providing verbal advice to other colleagues, either 
those who are more senior or in other departments, or directly to decision-makers individually 
or collectively. Likewise, ministers and senior officials use advisory content or processes to help 
educate and convince their cabinet colleagues, presidents and prime ministers, or other senior 
officials. The instrument of advice is used to raise considerations, to contest, to contextualize, 
to inform — or frustrate and delay — policy work at various stages of the policy process (Prasser 
2006; Craft 2015; Howlett 2012). 

An even broader view would reject the notion that it must be ‘government directed’ and rec- 
ognize that instruments of advice are now not only limited to those within the nodal position of 
government but also wielded by those ‘externals’ outside of government who seek to influence 
policy outcomes and behavior. Consultants, firms, and labor and non-governmental organiza- 
tions, both domestic and abroad, are regularly using policy advisory instruments to inform and 
influence policy processes at various times and regarding a broad range of policy issues (Craft and 
Halligan 2020; Stone et al. 2021). Whether presented through private conversations and brief- 
ing sessions or publicly available advice in the form of formal published recommendations and 
guidance, this is commonplace. In short, policy advice may not only lead to supply-and-demand 
effects outside government, but it can also have powerful market-like forces with respect to the 
supply and demand for policy work or programs and services within government. 


Substantive and Procedural Advisory Tools 


Policy advisory scholarship has typically made limited if any distinction around the types of advi- 
sory tools. Exceptions include attention to whether policy advice (the instrument) is written or 
verbal, whether it is formal or informal (meaning part of the formal decision-making system or 
a more informal ‘kitchen cabinet’ of advisors), or via advisory bodies like commissions, agents of 
legislatures, non-governmental advisors, or appointed political advisors who provide a spectrum 
of policy advice, including more partisan-political advice (Craft 2015; Craft and Halligan 2020). 

Looking to the procedural and substantive nature of policy advisory tools can help advance 
that effort. One of the overlaps noted earlier in relation to policy tools and advice was the dis- 
tinction and study of procedural and substantive aspects. As covered in greater detail in other 
chapters of this volume, substantive policy tools are ‘intended to directly affect the nature, types, 
qualities and distribution of goods and services provided in society’ (Howlett, 2000, p. 415). 
Procedural tools, in contrast, are aimed at altering policy processes rather than substance or 
‘intended to manage state-societal interactions in order to assure general support for government 
aims and initiatives’ (ibid., p. 412). The policy advisory literature has focused on substantive 
policy matters as the content or substance of advice and how that is generated and impacts policy 
while procedural aspects have focused on the processes by which policy advice is generated, 
brokered, and consumed in the policy process (Craft and Howlett 2012). While not identi- 
cal in their approach to the distinctions, they both capture the material effects or procedural 
implications of advisory activity. 

Policy advice and information as a policy tool have most often been characterized and studied 
as procedural policy tools. In fact, advisory-specific examples are often used to point out the way 
in which government can alter the actors, processes, activities, or purposes of advisory instruments 
with the intent of impacting policy behavior or outcomes (Craft and Halligan 2020; Howlett et al. 
2010a). Procedural nodality—related tools are ‘based on government information resources in order 
to attempt to alter the behavior of policy network members involved in policy making processes’ 
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(Howlett 2009, p. 26). Specific examples pertinent to this chapter include setting up special advisory 
bodies or commissions, opening or closing policy advisory processes to public consultation, and 
changing the composition or membership of key advisory bodies (Pierre 1998; Prasser 2006; Bali 
and Halpin 2021; Stark and Yates 2021; Craft and Halligan 2020). 

This again, however, speaks only to the ‘outward-facing’ dissemination view of information 
tools, which misses the internal applications. For instance, related to how governments set up 
advisory processes within government, including formal briefing systems that serve to control, 
coordinate, and structure who generates policy advice and how within the public sector, along 
with the types of advice and how it is or is not coordinated amongst disparate actors. Practical 
examples include cabinet decision-making processes and committee structures, interdepartmental 
committees, budget-making processes, and specific processes put in place by senior officials or 
ministers to govern how they are advised or engage with policy advice from their staffs or other 
actors (Craft 2015; Craft and Halligan 2020; Scott and Baehler 2010). 

The dominant focus on procedural information—based policy tools also misses some of the 
substantive tools and uses that are information based. Information-based substantive tools are 
utilized for eliciting desired policy outcomes by supplying a target audience with information; 
policymakers seek to influence their thinking, knowledge, or beliefs towards a particular behavior 
and thereby increase the attainment of desired policy goals (Weiss 2002). In short, substantive 
tools then serve to directly affect the demand or supply of goods and services provided in society 
(Howlett 2017, p. 99). 

Outward-oriented demand-side examples include reduced travel to certain destinations 
in the wake of government advice regarding public safety or public health policy advice: for 
instance, when governments issue travel advisories related to terrorism or security threats, or the 
significant amounts of public health advice issued by local, regional, and national health units/ 
governments during the COVID-19 pandemic. This had implications for a range of business 
risks and practices with impacts on spending and consumption. Likewise, policy advice by 
governments linked to the COVID-19 pandemic had production and supply effects related to 
the use of sanitizers, masks, vaccines, and testing products, not to mention the effects based on 
guidance about when and how to safely patronize businesses and manage employees working 
from home or at places of business. Internally, governments were advancing policy advice that 
had material consequences for the supply and demand of policy work as governments turned 
to pandemic policy challenges necessitating supply and demand shifts in the resources, number 
and type of staff, and broader policy work required to meet the public health, safety, economic, 
social, and international policy obligations of the pandemic (Boin et al. 2021). The logic carries 
forward to non-pandemic examples in which government policy advice can have the effect of 
prioritizing and (re)defining policy problems or options with one possible consequences being 
shifts in the resources allocated by the state or market actors. 


Here, There, and Everywhere 


At a high level, trying to make sense of the uses of advisory tools is facilitated by coupling 
together the substantive and procedural matters with Boston’s (1994) strategic and operational 
distinctions. The latter’s categorization also helps confront a persistent issue in policy instrument 
and advisory studies. As Howlett et al. (2018, p. 160) put it, 


Although policy instruments appear in all stages of the policy process, those affecting 
the agenda-setting, decision-making and evaluation stages of the policy process, 


while very significant and important in public management (Wu et al. 2010), have 
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traditionally not been considered in policy design research, which is centered on the 
formulation of tools and their mixes. 


Policy advice, too, has had a similar treatment, with the emphasis resting on policy advice in relation 
to policy formulation and decision-making activities (Craft and Wilder 2017). While this chapter does 
not go through each conjuncture of the policy-making process, Table 16.2 does provide examples, 
using the strategic and operational distinctions, coupled with the procedural and substantive elements 
outlined earlier and the audience, internal or external, to which advice is orientated. 

Front-end or agenda-setting instruments may involve using externally oriented policy advisory 
instruments to signal policy direction with implications for firms or sectors that may adjust in 
anticipation of government policy changes. For example, publication of a government white paper 
or ‘expert panel’ on competition policy or anti-trust policy may push firms or sectors to shift supply or 
impact demand. Published government advice may signal potential government activity in a sector 
or provide a set of guardrails that directs firms or citizens to the government’s preferred behavior 
or boundaries on policy issues. Internally, a range of agenda-setting policy advisory instruments 
can exist with clear links to the markets for policy work within government and to market effects 
outside it. Initial meetings between senior officials and ministers may involve advice about how 
approach certain policy files, advance key policy priorities, or recommendations about how to 
work with stakeholders or lead to changes in how initiatives are designed and (de)funded with 
market implications. Externally oriented advisory tools ofa procedural nature include setting terms 
of reference for public advisory bodies or processes and structuring public consultation topics or 
timelines with implications for the policy agenda or formulation activities. 


Table 16.2 Advisory Instrument by Audience, Function, and Policy Conjuncture 


Strategic ‘Front End’ Operational ‘Back End’ 
Internal External Internal External 

Substantive Policy advice Public health Implementation Public government 
that prompts advice advice from evaluation advice 
prioritization, (re) (e.g., advice on officials (e.g., published advice 
definition of a policy mask use, (e.g., briefings that regarding effective/ 
problem or objective safe travel, or recommend efficient service 
with implications for recommended working with/ providers) 
market dynamics business against a 

(e.g., public advice occupancy stakeholder, 
recommending capacity) firms, or 
new/revised community 
competition policy) partner) 

Procedural Establishing or Advice to establish Establishing Advice on how to 
managing internal or manage delivery units structure or managing 
government policy externally (e.g., to prioritize public- facing 
advisory processes oriented advisory implementation implementation or 

(e.g., advising on how processes or evaluation evaluation processes 
to set up briefing- (e.g., setting terms advice at the (e.g., recommending 
note and cabinet of refence center of favorable evaluators/ 
decision-making or public government) public members 
processes) consultation for evaluation 

timelines) committee) 
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Back-end procedural policy advisory instrument examples include the internal use of implemen- 
tation or progressing chasing units, which create advisory tools that prioritize delivery and utilize 
advisory instruments like performance dashboards or ‘stock takes’ on specific policy issues. These may 
lead to recommendations for recalibrations of approaches or resources reallocations inside or outside 
government (Lindquist 2006; Gold 2017). Externally, governments can be heavily involved in pro- 
cedurally structuring/managing evaluations or implementation working groups in setting terms and 
conditions and placing preferred participants in positions to influence implementation or evaluation 
advice or outcomes. Substantive advisory instruments could, for example, include inward-facing 
advice regarding implementation experiences with key stakeholders, firms, or other orders of gov- 
ernment that may influence future choices about whom to collaborate with. Externally, governments 
may publish advice about the effectiveness or efficiency of one service group over another 

These are just a few examples to illustrate that governments use advice as a tool beyond 
the confines of policy formulation and in ways that extend beyond the outward-facing public 
information campaign. Governments nodal positions given them power over many advisory 
processes that are internal as well as public facing. 


Conclusion 


It has been uncommon to think of advice as an instrument in the policy tools literature, with 
the information policy tool focus being on procedural tools, typically government informa- 
tion campaigns. The arguments presented in this chapter have been simple. Policy advice is an 
informational tool and has procedural and substantive applications and types. Advisory activity 
continues to be a significant element of what governments do, not only in terms of the public 
sector actors and organizations generating it but also how governments seek to structure its pro- 
cesses and use it strategically to influence a range of actors, both within and outside government 
(Hood 2006; John 2013). There is a pressing need to recognize the inward or internally facing 
suite of policy advisory tools that impact how governments advise and the types of advice that 
are privileged and constrained in policymaking. How advisory systems are set up within gov- 
ernment and the types of advice that are used, and when, are consequential to resource alloca- 
tions and policy workflows in government, both in policy formulation and beyond it. Likewise, 
governments externally facing advice can have important substantive and procedural effects on 
markets as noted in the examples provided. 

While it is easiest to think of policy advice as associated with formulation and design activi- 
ties, this chapter has argued that there are many examples in which agenda-setting, decision- 
making, or evaluation and implementation involve advisory tools. Policy advisory tools are 
apparent as governments seek to chart agendas, define policy objectives, and commit to policy 
implementation plans and programs and services. The broader suite of advisory tools at the front 
and back ends of policy points to a need for more research into the nature and types of tools and 
their policy implications. 

Similarly, recognizing the diversity of advisory tools and when they are put to work raises the 
issue of how these tools are combined in instrument ‘mixes’ (Howlett et al. 2018). Like policy 
advisory systems, instrument mixes reflect the reality that, in many instances, there is not a single 
instrument but several — with governments needing to think through how they are combined 
and interact. This resonates with advisory system scholarship, which has had a similar interest 
in understanding how the diverse sets of advice available to government impact governance and 
policymaking. While some attention has been paid to how external or partisan-political advice 
is coupled with that of public services, there remain several rich avenues of study to better 
understand how advisory instrument mixes operate and how governments can optimize them. 
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Classic definitions of public policy refer to government action and inaction — attributes readily 
apparent in the choice of commissions of inquiry and task forces as policy tools. These tools 
are unique for many reasons, not the least of which is that they can be simultaneously tools of 
both action and inaction. Temporary policy tools, which are often born out of crisis, live a short 
life and die a quick, predetermined, and often inglorious death; they can serve the purpose of 
inspiring government to action or providing cover for governments that do not want to pursue 
action. These tools are creatures of the executive branch of government, generally relate to 
policy review, and typically evade legislative scrutiny. This chapter outlines the general nature 
and purpose of commissions of inquiry and task forces as policy tools, covers the ways in which 
they contribute to policy formulation and other parts of the policy process, and uses an analytical 
framework focused on ideas, institutions, actors, and relations to highlight and assess their utility 
and value as policy tools. 


Introduction 


Crisis, scandal, and revelation are great catalysts in government affairs and policymaking. When 
they strike, governments are asked: How did this happen? Who was responsible? What was the 
government’s failure? What must be done to avoid future occurrences? Governments respond 
in many ways, particularly in the agenda-setting and formulation stages of the policy cycle, 
using the wide range of policy tools covered in this book. Policy makers have a wide range of 
advisory systems and options (Craft and Halligan, 2017); however, some policy conundrums are 
intractable and require exceptional consideration. In these cases, the policy tool often used is the 
commission of inquiry or task force. 

The structures and functions of commissions of inquiry and task forces have frequently been 
the subject of study for academics. Scholarship reaches back at least to the early twentieth 
century, looking at the unique role they play in politics and policy (Hodgetts, 1949, 1952, 1966; 
Courtney, 1969; Doern, 1967; Walls, 1969). Scholarship has tended to focus on single, in-depth 
case studies, classifications, and analysis of how inquires have evolved (Gorecki and Stanbury, 
1979; Drache and Cameron, 1985; Pross, Christie and Yogis, 1990; Litt, 1992; Jenson, 1994; 


208 DOI: 10.4324/9781003163954-21 


Tools for Structuring Policy Advice 


Wherrett, 1995; Timpson, 1999; Bradford, 1999/2000; Orsini, 2002; Ferguson and Wardaugh, 
2003; Manson and Mullan, 2003; Inwood, 2005; Clark and Trick, 2006; Scala, 2008). There 
is also some scholarship that examines and compares inquiries across time and a range of policy 
issues (Inwood and Johns, 2014; Johns and Inwood, 2018). 

Legal scholars have taken a particular interest in commissions of inquiry and task forces (Man- 
son and Mullan, 2003; Ratushny, 2009); so, too, have political scientists and others interested in 
public policy and public administration (Doern, 1967; Courtney, 1969; Walls, 1969; Bashevkin, 
1988; Schwartz, 1997; d’Ombrain, 1997; Bradford, 1999/2000; Stutz, 2008; Inwood and Johns, 
2014, 2016, 2018). The widespread and enduring interest in commissions of inquiry and task 
forces is evident. Scholarship has also emerged in other commonwealth and Westminster systems 
(for example, see Lauriat, 2010; Rowe and McAllister, 2006 on the UK; and Gilligan, 2002; 
Prasser, 1985, 1990, 1994, 2021 on Australia), and there is some comparative literature across 
jurisdictions (Marier, 2009) and on alternative forms (Eburn and Dovers, 2015). Only rarely 
have commissions of inquiry and task forces been compared directly, though (Aucoin, 1987). 
Our contribution is to conceptualize them as policy tools and examine their critical dimensions 
and utility in the pantheon of instruments available to governments related to policy. 

Traditional policy processes and tools can only go so far in addressing complex and vexing 
problems confronting the modern state and society. Anomalous conditions arise from time to 
time, which require fresh approaches outside the realm of traditional executive, legislative, and 
judicial policy toolboxes. Sometimes, the relatively slow-moving and often cumbersome appara- 
tus of government requires speeding up but can only do so outside the existing mechanisms for 
policymaking. Contested terrain may require an independent policy toolset where ideas can be 
tested, institutions can be rethought, different actors can be engaged, and new relations forged. 

This chapter focuses on commissions of inquiry and task forces as policy tools by addressing 
several key questions: Where do commissions of inquiry and task forces fit in the pantheon of 
policy tools available to governments? Do they matter to policymaking and formulation? Are they 
effective in bringing about change or in preventing change and sustaining the status quo? After 
providing some foundational definitions, the chapter outlines and applies a basic framework of ideas, 
institutions, actors, and relations for examining commissions and task forces as policy tools. 


Commissions of Inquiry and Task Forces: 
A Note on Nomenclature 


The terms commission of inquiry and task force require some explanation. These terms refer broadly 
to bodies which are extra-governmental but which are summoned into being by the executive 
branch of government. In constitutional monarchies, the term royal commission is often used for 
commissions of inquiry, though the former usage tends to delineate a more formal, larger, and 
more extensive form of inquiry than the latter term, which is also used. Commission of inquiry 
is used to describe a range of investigative tools, often with quasi-judicial features. The term 
task force originated in the United States Navy and is more common in democratic republics. 
It is essentially a special committee, typically composed of experts in a given field. Its job is to 
systematically study a particular problem, produce an evaluation of the causes, itemize options, 
and make recommendations based on the best of a set of possible solutions as determined by a 
set of standards. It is then up to the executive to act on the task force’s recommendations — or 
ignore them. The subject matter of task forces tends to be less formal and significant than that 
of commissions of inquiry, with some significant exceptions. 

These investigative tools are used extensively by all Westminster countries (e.g., Canada, the 
UK, Australia, and New Zealand). Examples number in the thousands and cover virtually every 
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policy field imaginable. Other countries besides Westminster nations have similar investigative 
models. For instance, the United States has the National Commission and Public Inquiry, such 
as the National Commission on Terrorist Attacks Upon the United States (also known as the 
9/11 Commission) and the Report on the Investigation into Russian Interference in the 2016 
Presidential Election (also known as the Mueller Report). Even though they are supposed to 
be nonpartisan and nonpolitical in nature, the acceptance and rejection of the results of com- 
missions of inquiry and task forces can be as political as any other decision made in any political 
institution. 

Commissions of inquiry are official inquiries into issues that affect the public. According to 
Gilligan (2002), they originated from the British monarchy, which used the royal prerogative 
to order investigations. However, because of the broad nature and uses of the commission of 
inquiry and their complexity, differing mandates, purpose, duration, and size, it is challenging 
to nail down a single comprehensive definition for them. 

Nevertheless, commissions of inquiry can be referred to as official reviews authorized by 
governments to investigate an issue or event, understand the cause of the issue or event, and 
subsequently make recommendations (Inwood and Johns, 2014). Categorizations have been 
undertaken that generally sort them into policy-investigative and legal-judicial types. 

Commissions of inquiry are any working group mandated by the government but subor- 
dinate to it that is given the job of studying a particular policy or program to produce a non- 
binding report and recommendation (Marier, 2009). Although they are a policy tool deployed 
by the executive branch, they are not themselves part of the executive, legislative, or judicial 
branches of government; thus, they are not directly answerable to any political master. They 
have been described as having varying degrees of impact on policy change and formulation as 
“independent creature[s] existing in a shadowy realm of extra-governmental institutions that are 
nonetheless public” (Inwood and Johns, 2014: 8). 

Task forces are also a type of public inquiry and policy exploration undertaken by govern- 
ments, typically established by the executive. Task forces are temporary organizations, often 
involving multiple departments and agencies and levels of governments created to solve a par- 
ticular problem. According to Roderick Macdonald, task forces are non-statutory forms of 
inquiries established by a minister seeking to advance policy objectives in their department, 
ministry, or agency (2011). They are a more formal ad hoc committee, more limited in scope 
and duration, and are generally more specialized. Task forces are less formal and less independent 
than commissions of inquiry. As a result, they do not enjoy the power to subpoena witnesses, 
take evidence under oath, and request documents like the commission of inquiry (Macdonald, 
2011; Government of Canada, 2017). Simply put, task forces are policy tools created to investi- 
gate and inquire into issues and explore policy options; they can be made up of members of the 
public service, experts, academics, or members of the public (Macdonald, 2011). 

Generally, commissions of inquiry are reserved for more significant policy problems than 
task forces. But at times, the difference between them is almost nonexistent as some inquiries 
have been used to tackle relatively small-scale specific issues while some so-called task forces 
were actually commissions. Commissions of inquiry and task forces are used as policy tools 
in many jurisdictions. In other non-Westminster countries, there are several different types of 
public inquiries. In the case of the United States, for instance, the major ones are the Presi- 
dential Commission and the Congressional Advisory Commission. As its name suggests, the 
Presidential Commission is ordered by the president as the head of the executive branch under 
the Federal Advisory Committee Act to investigate a specific issue, event, or policy. The act 
endows presidential commissions with limited powers of investigation and requires the president 
to give a formal response within 12 months of a presidential commission report being submitted 
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(Prasser, 2021). In contrast, the Congressional Commission has five characteristics that differen- 
tiate it from a Presidential Commission. The former “(1) is established by Congress, (2) exists 
temporarily, (3) serves in an advisory capacity, (4) is appointed in part or whole by Members of 
Congress, and 5) reports to Congress” (Congressional Research Service, 2021: 1). 

Many of the same motivations for establishing commissions of inquiry and task forces 
extend across various jurisdictions, whether Westminster-style or republic. They provide 
highly visible forums to contend with difficult issues while bringing together expertise that 
may be absent in the traditional branches of government. Longer, deeper examinations can be 
undertaken. Non-partisanship can make recommendations more politically salient and sale- 
able both within government and with the broader public. The criticisms that these endeavors 
are expensive, and the perception that government may be dodging its responsibility to tackle 
tough issues is common. So, too, are the charges that findings could have been arrived at 
through a reasonable effort by the executive or legislative branches and/or are mostly ignored 
in the policy process. 


Commissions of Inquiry and Task Forces as 
Policy Tools in the Policy Process 


Commissions of inquiry and task forces are unique policy tools. Their temporal dimension 
(short lived), place (outside the realm of regular institutions such as parliamentary-cabinet gov- 
ernment, the courts, etc.) and power (special investigatory) make them so. In this section, we 
consider the following: Where do commissions of inquiry and task forces “fit” in the pantheon 
of policy tools? And what does the literature say about them as policy tools? 

Policy instruments, also called “policy tools” or “governing instruments,” covered in detail 
in other chapters of this book, are the means by which governments can choose to reach their 
policy goals. There are a large variety of instruments and reasons for their selection, and they are 
important to all stages of the policy cycle. 

Table 17.1 locates commissions of inquiry and task forces in the policy cycle. Noteworthy is 
that the problem definition and agenda-setting stage reveal that “traditional” policy instruments 
are not up to the task of tackling the given policy problem; this results in the government 
choosing the extraordinary tool of a commission of inquiry or task force, which can also be 
considered to serve as agenda-setting and problem-definition tools as described by Prasser 
(1994). 


Table 17.1 Commissions of Inquiry and Task Forces as Policy Instruments in the Policy Cycle 


Policy Cycle Commissions of Inquiry and Task Forces 

Problem Determines that there are no “traditional” instruments with which to solve a 
definition and problem, and extraordinary measures need to be considered as the problem 
agenda-setting is unlikely to disappear from the government’s agenda 

Policy Involves the government choosing the commission of inquiry or task force to 
formulation supplant the government in the process of policy formulation 

Policy Implementation of the commission of inquiry or task force recommendations 
implementation is an entirely discretionary act on the part of the government 

Policy evaluation No formal role; in rare instances, former commission of inquiry or task 


force members critically assess the implementation or lack thereof of their 
recommendations 
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These tools are a part of the policy formulation stage of the policy cycle. It is their funda- 
mental role to devise recommendations that represent the most promising policy formulation for 
the resolution of the policy problem. Once selected, several important features outline the scope 
and function of commissions of inquiry and task forces as policy tools. These include mandate 
and scope, typically outlined in a “terms of reference” time frame and pressures for completion; 
number of commissioners; staff; financial resources; research capacity and character; capacity for 
public participation; writing the final report; and the relationship between the inquiry and the 
government. These important features relate to the design, operation, and outcomes of these 
tools in the policy formulation stage. 

Some of the most successful commissions of inquiry and task forces have been those in which 
successive governments adopt and continue to work to implement recommendations. Recom- 
mendation trackers are now being used regularly by civil society organizations and academics to 
gauge implementation progress after governments formally accept recommendations made by 
commissions of inquiry and task forces. These accountability mechanisms increase the utility of 
these temporary policy tools in policy formulation. 

The commission of inquiry or task force has no relationship to the implementation stage of 
the policy cycle. This function is reappropriated to the government. Once the final report is 
handed over, the commission of inquiry or task force has no say whatsoever in the implementa- 
tion of its recommendations. It is effectively dead as a policy tool. It is up to the government 
to decide what to do and determine if the prescriptions proposed are politically, economically, 
and culturally acceptable and desirable. Is there the political will or government commitment 
to policy change or policy formulation? Are the recommendations for policy change or new 
policy formulation a priority for the government? Will they endure if there has been a change 
in government? 

Similarly, the commission of inquiry or task force is not a tool of evaluation insofar as the 
implementation of its own recommendations are concerned. Occasionally, a commission of 
inquiry or task force member will comment publicly on the implementation (or lack thereof ) 
of its recommendations, usually critically. However, this is relatively rare, and even then, only 
occasionally would this type of evaluation be influential on the policy process after the commis- 
sion of inquiry or task force has completed its utility as a policy tool. 

The selection of a commission of inquiry or task force as a policy tool is based on a 
variety of factors. These include the cost of action versus inaction, the size of the impacted 
community, public opinion, political direction, ideology, timing, and the need for creativity 
related to policy change. 

As outlined in other chapters in this book, various policy theories seek to explain the choice 
of policy tool, and these can be applied to commissions of inquiry and task forces. For example, 
it is argued in classic neo-institutional theory that their use is limited by previous instrument 
choice or the level of experience with the instrument by the institutions with authority for 
making instrument choice. This is classical “path dependency” in neo-institutionalism, in which 
consecutive governments continue to create commissions of inquiry and task forces to try to 
address complex policy problems. Classical instrument choice theory also posits that policy 
makers may choose commissions of inquiry or task forces to move from least coercive action to 
more coercive action through classical incrementalism. A fundamental assumption under pub- 
lic choice theory is that these instruments are selected based on an individual political leader’s 
objectives of improving their own political position; commissions of inquiry and task forces are 
seen as less intrusive instruments when seeking support or when politicians or governments do 
not want unnecessary backlash. In this regard, shuffling an issue off to a commission of inquiry 
or task force can be seen as a salubrious choice. 
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Commissions of inquiry and task forces can also be conceptualized as indirect policy tools. 
They are information-based instruments intended to shed new light on an issue and/or perhaps 
provide an educative function, influencing ideas, expertise, and argument. This recognizes that 
human behavior is based on knowledge, beliefs, and values; perhaps if these can be changed, 
behavior will change. This is the least coercive of all policy instruments: there is no obligation 
to act and no supplementary inducement or penalty. A problem, though, is that it is hard to 
measure the efficiency or effectiveness of government money spent on an instrument for “infor- 
mation” or “public education” in the form of a commission of inquiry or task force. 

In modern democratic systems of government, the choice of policy tools is constrained by 
the dominant ideational parameters of a liberal capitalist democracy within which contestation 
over the role of the state and markets is of paramount importance. Institutionally, a constitutional 
monarchy or republic, Charter of Rights and Freedoms, federalism, parliamentary-cabinet govern- 
ment, professional public service, independent judiciary, and competitive party system influence 
the choice of policy instruments. The key actors who influence the choice of policy tools include 
elected politicians, appointed public servants, the judiciary, the public, political parties, interest 
groups and social movements, think tanks and research organizations, and the media. The choice 
of policy tools is also influenced by the set of relations that dominate in the political, economic, and 
social/cultural domains of the society. Relations can be both inside and outside government itself 
and can be constituted formally (as with the hierarchy of the public service or the constitutional 
division of powers) or informally (as in the professional trust ties, common educational/technical 
backgrounds, social connections, and other personal affiliations of key actors). 

Some combination of ideas, institutions, actors, and relations needs to coalesce to bring 
about the choice of commissions of inquiry or task forces as the preferred policy tool for a given 
policy problem. Using a framework of ideas, institutions, actors, and relations has been shown to 
be a very valuable approach when comparing the unique dimensions of these policy tools across 
many policy areas and eras (Inwood and Johns, 2014). This four-part framework also allows for 
the detailed analysis of the unique internal features of commissions of inquiry and task forces 
themselves and a focus on the outcomes that flow from these policy tools. 


Analyzing Commissions of Inquiry and Task Forces as Policy Tools 


Ideas 


The ideational framework influencing the choice of any policy tool is multi-faceted. Where 
commissions of inquiry and task forces are concerned, precedential knowledge, information, and 
fact are open to contestation. As Inwood and Johns explain, ideas refer to democratic and political 
beliefs that influence the behavior of decision-makers and the action they choose (2014). Ideas 
include knowledge (i.e., factual evidence) acquired through empirical research, systematic pro- 
gram and policy review, evaluation and assessment of progress, political manifestos, and electoral 
promises, as well as organizational analysis. In addition, informed, educative, and expert-driven 
opinions are also considered in the realm of ideas, as are ideas about the state versus the market 
and political ideology. Lastly, ideas can also originate from societal beliefs such as social justice, 
fairness, equality, equity, diversity, and due process (Johns and Inwood 2104: 28-33). 

Typically, the origin, context, and use of ideas are in question when choosing commissions 
of inquiry and task forces as the policy tool to tackle policy issues. Important societal ideas of 
modern democratic societies also include legitimacy, transparency, accountability, efficiency, 
and most importantly, democracy. Choosing these tools, whether as a politically calculated deci- 
sion to delay, shift blame, or appear transparent and accountable or simply for a policy-learning 
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reason (Marier 2009; Stark 2019), commissions of inquiry and task forces fulfill many of the 
requirements of democratic society. 

As noted earlier, an important reason to commit to a commission of inquiry or task force is 
that the regular legislative process or public service is not suited to perform the sort of policy 
in-depth analytical research, formulation, and critique often required. Commissions of inquiry 
and task forces can fulfil the efficiency imperative required in investigating disasters and crises 
where the regular public service may not be able. Old thinking conditioned by path dependency 
or bureaucratic inertia can be countered with a fresh analysis of the ideational landscape. Con- 
versely, existing dominant ideas can be reinvigorated or defended by a commission of inquiry 
or task force expressly created to sustain the ideational status quo. The ability of commissions of 
inquiry and task forces to uncover the causes of issues, crises, and disasters and make recommen- 
dations free of the political ideology of a given government is inherently an important reason 
they are utilized. But so, too, is the imperative to confirm or legitimize the political ideology of 
a given government. 

Drawing these conclusions is often dependent on the level of analysis. If a major oil spill 
results in a commission of inquiry, one level of analysis may be that the technological failures 
behind the disaster ought to be investigated. But a broader level of analysis might refer to ques- 
tions about the dominant capitalist-driven reliance on fossil fuels that contribute to climate 
crisis. The ideational terrain therefore influences the decision to use commissions of inquiry and 
task forces as policy tools to investigate, how deeply, and from what perspective, 

The nature of the policy also contributes to the idea to utilize commissions of inquiry and 
task forces instead of other policy tools. In a crisis (such as a pandemic), tragedy or disaster 
(human-induced or natural), or systemic institutional failure (as in the case of the National 
Inquiry into Murdered and Missing Indigenous Women and Girls in Canada), existing political 
and administrative institutions can unwittingly hinder a proper investigative process to address 
various types of crises (Schwartz and McConnell, 2009; Rena and Christensen, 2020). In sum- 
mary, the idea to create commissions of inquiry and task forces instead of allowing the legislature 
or public service to do the same function depends on multiple factors, including the state of 
ideas in play. 

Finally, the mere act of establishing commissions of inquiry and task forces creates an institu- 
tion in which ideational contestation over policy, problem definition, agenda-setting, and policy 
solutions is brought into existence (Johns and Inwood, 2014). The fact that commissions of 
inquiry and task forces are often composed of members of the academic community, legal com- 
munity, and various societal enterprises that are involved in determining which type of research 
and evidence to be considered in finding, reporting, and making recommendations are other 
ideational factors of importance. 


Institutions 


When the traditional institutions and avenues of policy change have been exhausted, or when 
governments want to project a sense of legitimacy, accountability, and transparency to the public, 
they often turn to commissions of inquiry and task forces. Institution here refers to the formal 
political system, which includes the constitution and federalism (in some countries) and the 
executive, legislative, and judicial branches of government. All these avenues of policy decision- 
making can be considered institutions. Similarly, in the context of decision-making, commis- 
sions of inquiry and task forces are considered temporary policy tools that exist alongside these 
other institutions. The formal institutions, primarily the executive branch, set the parameters for 
each commission of inquiry or task force. While the executive, legislative, and judicial branches 
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of government are the primary homes of policy decision-making, their inadequacies necessitate 
an alternative. Creating a commission of inquiry or task force opens up opportunities for the 
other traditional institutions to formulate and change policy in ways perhaps impossible other- 
wise to achieve. 

The internal features of commissions of inquiry and task forces contain an array of adminis- 
trative and political characteristics. They can be any size and duration, as the terms of reference, 
mandate, and scope are entirely determined by the government (Canada, PCO, 2022). The 
decision to create a commission of inquiry or task force is very much the product of delibera- 
tions by the executive, frequently driven at the national level by a prime minister or president 
and at the sub-national level by a premier or governor. The architecture of the actual policy 
tool is usually the product of internal deliberations involving cabinet and senior public servants. 
Whether the government is trying to use a commission of inquiry or task force as a political 
management tool to divert attention or a policy learning tool to innovate policy will determine 
its features (Mairer, 2009; Stark, 2019). For instance, if the government is trying to use a policy 
investigative commission of inquiry or task force as a delay and blame-shifting mechanism, the 
scope may be expansive to ensure that enough time is used up or the final report is not published 
until after an election. However, in investigative commissions of inquiry or task forces, the scope 
is likely to be narrow. 

What should a commission of inquiry or task force look like? There is no one correct answer. 
Some are modest affairs with a single chairperson, small staff, and modest budget. Others are 
comparatively speaking gargantuan affairs, with as many as 20 commissioners, an army-size staff 
and truckloads of financial resources. Overall, the size of the commission, staff, mandate, and 
financial resources is determined by the issue at hand and the extent to which it is politicized. 
The size also varies around the world; for instance, in the UK, the size can range from one to 
twenty commissioners, with an average membership of ten (Lauriat, 2010: 12). In Canada, many 
one-person commissions have been created while the largest was composed of thirteen commis- 
sioners (Inwood and Johns, 2014). 


Actors 


Actors refer to all state political, administrative, and societal policy actors. Political actors refer 
to the presidents, prime minister, governors, premiers, cabinet ministers, and political staff. 
Administrative actors include the public service. Societal actors include all types of stakehold- 
ers that are interested in shaping public policy or holding government accountable, including 
individuals, interest groups, social movements, unions, academics, experts, the public, and the 
media (Inwood and Johns, 2016: 387). All the actors both contribute to and have a stake in the 
successes or failures, recommendations, and overall outcomes of commissions of inquiry and 
task forces. 

Besides the state actors that ordered the commission of inquiry or task force, the actors 
conducting it are also influential. Who the chairperson, commissioners, staff, and researchers 
are has a profound impact. Being human, they all bring their biases and prejudices to the sub- 
stantive issue under investigation (Lauriat, 2010: 12). The chairperson and commissioners are 
historically from a small pool of elite communities such as judges, retired cabinet members, or 
prominent business people, and this has not changed over time (Lauriat, 2010:13). The staff who 
serve administrative functions are usually seconded from law firms, academia, or various private 
sectors along with public servants (Johns and Inwood, 2014: 37). 

As noted earlier, the creation of commissions of inquiry and task forces is determined by 
actors in the executive institutions of government. But once established, their work is potentially 
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subject to the influence of a large body of other actors. Contestation over the terms of reference, 
size, duration, and staffing is a process largely internal to the government amongst political and 
administrative actors. Subsequent contestation over the scope of the investigative aspects of the 
commission of inquiry or task force is a two-headed process. First, contestation occurs within 
the commission of inquiry or task force itself as the commissioner(s), staff, and expert advisors 
jockey for influence and the ability to place their stamp on the process. According to Barbara 
Lauriat, the chair and other commissioners (if there are more than one) and the staff members 
profoundly impact all aspects of the proceedings and are likely to bring their biases and prejudice 
to the substantive issues under consideration (Lauriat, 2010: 12). 

Second, contestation occurs externally as societal actors seek access to the commission of 
inquiry or task force and attempt to impress their views upon its key decision-makers. The 
extent of these actors’ influence is often conditioned by the rules of the game; some commissions 
of inquiry and task forces are broadly consultative and allow for numerous expressions of interest 
to be heard. Others are more selective. 


Relations 


Both formal and informal relations affect commissions of inquiry and task forces, like many 
other political/policy tools, including in their establishment. The formal relations flow from 
the initial choice to use these policy instruments outlined under the legislation that grants them 
authority, but also in the terms of reference that outline the scope, mandate, and relations that 
will formally underpin a commission of inquiry or task force. 

Informal relations also matter. For instance, some are launched because of the personal 
relationship between political actors, who may know well each other’s policy proclivities, or 
between state and non-state actors who are ideologically aligned. Given that the head of the 
executive branch is primarily responsible for determining the need for and the personnel at 
the top of commissions of inquiry or task forces, this is not surprising; good relations are often 
a precondition of the choices made by heads of government. In addition, the relationship 
between the commissioner(s) and the staff/researchers is also of significance. Some are smooth 
and collegial; others are plagued by virtual civil war (Inwood and Johns, 2016: 396). 

Furthermore, policy entrepreneurs within the commission of inquiry or task force and how 
they relate to others are also factors that affect their outcome and the probability of policy change. 
Policy entrepreneurs who take the policy window of opportunity of commissions of inquiry 
or task forces to peddle their agenda and attempt to influence its direction toward their policy 
interest must be aware that others may have different designs. The art of managing and massaging 
relations, therefore, can be very significant and involve commissioners, staff, researchers, and/or 
various interest groups and policy networks involved in the process. Relations definitely matter 
related to commissions of inquiry and task forces as policy tools (Inwood and Johns, 2014). 


Conclusion 


Australian professor Scott Prasser argues that hardly a day passes in Australia without a public 
inquiry or royal commission being appointed, a report being released, or someone calling for an 
inquiry (Prasser, 2021). The same could be said in virtually every country where these policy 
tools are employed. Commissions of inquiry and task forces are commonly used yet unique as 
tools in government’s policy toolbox (Howlett et al., 2009). Used for centuries, they remain 
available for governments of all political ideologies. They appeal to political leaders and deci- 
sion-makers due to their ability to result in action, inaction, and everything in between. 
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For as long as governments have been using commissions of inquiry and task forces as policy 
tools, they have been assessed as having both strengths and weaknesses inherent to their features 
but also in comparison to other policy tools covered in this book. They are assessed by some as 
legitimate tools for in-depth research, analysis, wisdom, judgment, and recommendations; in the 
context of representative democratic policy making institutions with withering legitimacy, they 
often lend cache to governments apparently bereft of policy options. But they have also long 
been criticized as being expensive, difficult to control, too lengthy, too slow, elitist, and ineffective 
except as tools for delaying, obfuscating, or avoiding policy decisions and actions. 

Notwithstanding whether one loves, hates, or is indifferent to commissions of inquiry and 
task forces, they have endured. On balance, they are viewed as a choice of policy tools for 
generating policy recommendations for policy change. They remain policy tools that bring 
state and societal actors and interests together to attempt to collectively understand, analyze, 
address, and design better policy approaches and solutions. They are tools that embrace the 
complexity of policy issues and bring together key elements of policy formulation: ideas, 
institutions, actors, and relations. Commissions of inquiry and task forces are likely to remain 
important and appealing as policymakers grapple with the increasing complexity of many 
policy issues. 

In a policy context where evidence, facts, equity, and values are increasingly important and 
where representative democracies continue to see their legitimacy challenges and deliberative forms 
of democracy viewed as superior and necessary, commissions of inquiry and task forces may become 
even more important policy tools. They speak to the classic definition of policy as government 
action and inaction as unique, temporary policy tools. They are often a response to crisis, function 
for a set and brief period, and terminate on a predetermined schedule. Consequently, they can pro- 
vide governments with the rationale to take action or the rationale to avoid it, all the while acting at 
the behest of the executive branch and largely avoiding the scrutiny of legislatures. This chapter has 
examined the general nature and purpose of commissions of inquiry and task forces as policy tools 
and their contribution to the policy process, using an analytical framework of ideas, institutions, 
actors, and relations. Crisis, scandal, and revelation will always be with us; so, too, apparently, will 
commissions of inquiry and task forces. 


References 


Aucoin, Peter. 1987. “Royal Commissions and Task Forces as Mechanisms of Program Review.” The 
Canadian Journal of Program Evaluation, 2(2) (Fall): 1. 

Bashevkin, Sylvia. 1988. “Does Public Opinion Matter? The Adoption of Federal Royal Commission and 
Task Force Recommendations on the National Question, 1951-1987.” Canadian Public Administration, 
31(3), 390-407. 

Bradford, Neil. 1999/2000. “Writing Public Philosophy: Canada’s Royal Commissions on Everything,” 
Journal of Canadian Studies, 34, 4 (Winter), 136-167. 

Canada, Privy Council Office, 2022. About Commissions of Inquiry. https://www.canada.ca/en/privy- 
council/services/commissions-inquiry/about.html 

Clark, Ian D. and David Trick. 2006. “Advising for Impact: Lessons from the Rae Review on the Use of 
Special Purpose Advisory Commissions.” Canadian Public Administration, 49, 2 (June), 180-195. 

Congressional Research Service. 2021. Congressional Commissions: Overview and Considerations for 
Congress. https://fas.org/sgp/crs/misc/R40076.pdf 

Courtney, John C. 1969. “In Defence of Royal Commissions.” Canadian Public Administration. 12, 2 (Fall), 
198-212. 

Craft, Jonathan and John Halligan. 2017. “Assessing 30 Years of Westminster Policy Advisory System 
Experience.” Policy Sciences, 50(1), 47—62. 

Doern, G. Bruce. 1967. “The Role of Royal Commissions in the General Policy Process and in Federal- 
Provincial Relations.” Canadian Public Administration, 10, 4 (December), 405-523. 


217 


Gregory J. Inwood and Carolyn M. Johns 


d’Ombrain, Nicholas. 1997. “Public Inquiries in Canada.” Canadian Public Administration, 40 1 (Spring), 
86-107. 

Drache, Daniel and Duncan Cameron. 1985. The Other Macdonald Report: The Consensus on Canada’s Future 
That the Macdonald Commission Left Out. Toronto: James Lorimer. 

Eburn, Michael and Stephen Dovers. 2015. “Learning Lessons from Disasters: Alternatives to Royal Com- 
missions and Other Quasi-Judicial Inquiries.” Australian Journal of Public Administration, 74(4), 495-508. 

Ferguson, Barry and Robert Wardaugh. 2003. ““Impossible Conditions of Inequality’: John W. Dafoe, the 
Rowell-Sirois Royal Commission, and the Interpretation of Canadian Federalism.” Canadian Historical 
Review, 84, 4 (December), 551-583. 

Gilligan, George. 2002. “Royal Commissions of Inquiry.” Australian and New Zealand Journal of Criminology, 
35(3), 289-307. 

Gorecki, Paul K. and W.T. Stanbury, eds. 1979. Perspectives on the Royal Commission on Corporate Concentra- 
tion. Toronto: Butterworths. 

Government of Canada. 2017. Fact Sheet on Inquiries. www.rcaanc-cirnac.gc.ca/eng/1451952560307/ 
1534527794601 

Hodgetts, J.E. 1949. “Royal Commissions of Inquiry in Canada.” Public Administration Review, 8(4), 22-29. 

Hodgetts, J.E. 1952. “The Role of Royal Commissions in Canadian Government.” Proceedings of the Third 
Annual Conference of the Institute of Public Administration of Canada, 1951: 351-376. 

Hodgetts, J.E. 1966. “The Role of Royal Commissions in Canadian Government.” In J.E. Hodgetts and 
D.C. Corbett, eds. Canadian Public Administration: A Book of Readings. Toronto: Macmillan, 471—483. 

Howlett, Michael, Anthony Perl and M. Ramesh. 2009. Studying Public Policy: Policy Cycles and Policy Sub- 
systems. Oxford: Oxford University Press. 

Inwood, Gregory J. 2005. Continentalizing Canada: The Politics and Legacy of the Macdonald Royal Commis- 
sion. Toronto: University of Toronto Press. 

Inwood, Gregory J. and Carolyn M. Johns. 2014. “Why Study Commissions of Inquiry?” In Commissions 
of Inquiry and Policy Change. Toronto: University of Toronto Press, 1—19. 

Inwood, Gregory J. and Carolyn M. Johns. 2016. “Commissions of Inquiry and Policy Change: Compara- 
tive Analysis and Future Research Frontiers: Commissions of Inquiry and Policy Change.” Canadian 
Public Administration, 59(3), 382—404. 

Jenson, Jane. 1994. “Commissioning Ideas: Representation and Royal Commissions.” In Susan Phillips, ed. 
How Ottawa Spends 1994-95: Making Change. Ottawa: Carleton University Press, 39-69. 

Johns, Carolyn M. and Gregory J. Inwood. 2014. “Theories of Policy Change and a Four-Part Theoretical 
Framework for Comparative Analysis.” In Commissions of Inquiry and Policy Change. Toronto: University 
of Toronto Press, 20—48. 

Johns, Carolyn M. and Gregory J. Inwood. 2018. “Commissions of Inquiry and Policy Analysis.” in Policy 
Analysis in Canada, Laurent Dobuzinskis and Michael Howlett, eds. Policy Press Scholarship Online, 
233. 

Lauriat, Barbara. 2010. ““The Examination of Everything’: Royal Commissions in British Legal History.” 
Statute Law Review, 31, 24—46. 

Litt, Paul. 1992. The Muses, the Masses and the Massey Commission. Toronto: University of Toronto Press. 

Macdonald, Roderick Alexander. 2011. An Analysis of the Forms and Functions of Independent Commissions of 
Inquiry (Royal Commission) in Canada. Montreal: McGill Faculty of Law. 

Manson, Allan and David Mullan, eds. 2003. Commissions of Inquiry: Praise or Reappraise? Toronto: Irwin 
Law. 

Marier, Patrik. 2009. “The Power of Institutionalized Learning: The Uses and Practices of Commissions 
to Generate Policy Change.” Journal of European Public Policy, 16(8), 1204-1223. 

Prasser, Scott. 1985. “Public Inquiries in Australia: An Overview.” Australian Journal of Public Administration, 
44(1), 1-5. 

Prasser, Scott. 1990. “The Fate of Inquiries: Will Fitzgerald be Different?” In S. Prasser, R. Wear and J. 
Nethercote, eds. Corruption and Reform: The Fitzgerald Vision. St Lucia: University of Queensland Press, 
98-122. 

Prasser, Scott. 1994. “Royal Commissions and Public Inquiries: Scope and Uses.” In P. Weller, ed. Royal 
Commissions and the Making of Public Policy. Melbourne: Macmillan, 1-21 and 267-287. 

Prasser, Scott. 2021. Public Inquiries: An In-Depth History. www.publicinquiries.com.au/indepth_history. 
htm [Accessed August 11, 2021]. 

Pross, A. Paul, Innis Christie and John A. Yogis, eds. 1990. Commissions of Inquiry. Toronto: Carswell. 

Ratushny, ed. 2009. The Conduct of Public Inquiries: Law, Policy, and Practice. Toronto: Irwin Law. 


218 


Tools for Structuring Policy Advice 


Rena, Helge and Johan Christensen. 2020. “Learning from Crisis: The Role of Enquiry Commissions.” 
Journal of Contingencies and Crisis Management, 28(1), 41—49. 

Rowe, Mike and Laura McAllister. 2006. “The Roles of Commissions of Inquiry in the Policy Process.” 
Public Policy and Administration, 21(4), 99-115. 

Scala, Francesca. 2008. “Feminist Ideals vs. Bureaucratic Norms: The Case of Feminist Researchers and 
the Royal Commission on New Reproductive Technologies.” In Yasmeen Abu-Laden, ed. Gendering 
the Nation-State: Canadian and Comparative Perspectives. Vancouver: UBC Press. 

Schwartz, Bryan. 1997. “Public Inquiries.” Canadian Public Administration, 40, 1 (Spring), 72-85. 

Schwartz, Robert and Allan McConnell. 2009. “Do Crises Help Remedy Regulatory Failure? A Com- 
parative Study of the Walkerton Water and Jerusalem Banquet Hall Disasters.” Canadian Public Admin- 
istration, 52(1), 91-112. 

Stark, Alastair. 2019. “Policy Learning and the Public Inquiry.” Policy Sciences, 52(3), 397—417. 

Stutz, Jeffrey R. 2008. “What Gets Done and Why: Implementing the Recommendations of Public Inqui- 
ries.” Canadian Public Administration, 51(3), 501-521. 

Timpson, Annis May. 1999. “Royal Commissions as Sites of Resistance: Women’s Challenges on Child 
Care in the Royal Commission on the Status of Women.” International Journal of Canadian Studies, 20, 
123-148. 

Walls, C.E.S. 1969. “Royal Commissions: Their Influence on Public Policy’ Canadian Public Administration, 
12 (Fall), 365-371. 

Wherrett, Jill. 1995. “The Research Agenda of the Royal Commission on Aboriginal Peoples.” Canadian 
Public Administration, 38, 2 (Summer), 272-282. 


219 


18 


COLLABORATION AS A POLICY 
TOOL IN THE POLICY PROCESS 


Saba Siddiki, Jonathan E. Beagles and Nicholas Oesterling 


Collaborative forms of governance are increasingly recognized and studied as important features 
of modern public policy processes. Collaborative governance is akin to traditionally conceived 
policy tools (e.g., regulation, taxation, information dissemination) insofar as it is a mechanism 
through which governments pursue policy-relevant actions toward the attainment of public 
goals. In this chapter, we present collaborative governance as a policy tool, identifying (1) fac- 
tors that contribute to its engagement in policymaking, (2) ways it is structured through pub- 
lic policy, and (3) ways that collaborative governance manifests following its engagement and 
design, as well as factors contributing to sustained collaboration and collaboration efficacy. In 
doing so, we showcase how collaborative governance as a policy tool situates in different stages 
of the public policy process. 


Introduction 


Questions of how governments address public problems and deliver public goods and services 
are central to the study of public policy. Fundamentally, these “how” questions are aimed at fur- 
thering understanding of the tools with which governments do their work in accordance with 
broader policy objectives. Over the last several decades, policy scholars have dedicated substan- 
tial attention to the study of policy tools, contributing important insights about generic types 
of tools predominantly used by governments (Salamon 2002), factors informing tool choice 
(Linder and Peters 1989), and the behavioral assumptions associated with different types of 
policy tools (Schneider and Ingram 1990), among other topics. As policy practice has evolved, 
scholarship on policy tools has expanded to capture the widening array of tools deliberated, 
designed, and engaged in various stages of the policy-making process. 

In the 1990s and into the early 2000s, scholarship on policy tools underwent a notable 
shift in reaction to growing recognition of governments’ reliance on networks of public and 
private policy stakeholders in policy formulation and implementation (Howlett 2014). Policy 
scholars turned their attention to understanding increasingly “networked” or “collaborative” 
forms of governance in the context of which policymakers were seen less as directing and 
deciding on policy and more as steering processes in which multiple stakeholders were 
interacting to deliberate what governments should do and how they should do it (Ansell and 
Torfing 2016). 
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As research on collaborative governance has developed since this time, policy scholars have 
contributed important insights regarding conditions under which collaborative governance is 
engaged in the public policy process, how collaborative governance is structured through public 
policies, and how collaborative governance manifests once it is decidedly engaged and described 
through public policies. Further, this research also highlights factors that contribute to sustained 
collaboration and factors that enable collaborative success. In this chapter, we draw on extant 
research on collaborative governance to discuss each of these topics while positioning collabora- 
tive governance as a kind of policy tool. 


Engaging Collaboration as a Policy Tool in the Public Policy Process 


Collaborative governance has previously, and aptly, been described by Scott and Thomas (2017) 
as a toolbox collectively utilized and employed by multiple policy actors to achieve desired policy 
goals. These goals can range from improving the quality of policy outputs to increasing policy 
actor or action legitimacy, achieving economies of scale in public service provision, bridging 
hierarchies, and/or expanding the diversity of issues to which governments can respond. 

The decision to collaborate, and the form a collaborative governance arrangement takes, 
are often responses to the perceived nature of wicked public problems and the complex system 
environment in which actors are embedded. Scholars describe these environments in various 
ways. Yet there appears to be general agreement regarding the importance of five broad types 
of conditions on the initiation and design of collaborative processes (Bryson, Crosby, and Stone 
2006, 2015; Emerson and Nabatchi 2015). These types of conditions are described next. 


Resource and Service Conditions 


The condition(s) of the resources and/or services present within the environment of potential 
collaborators can serve as a baseline from which decisions to collaborate are made. Resource 
conditions broadly encompass natural resources or human-made resources. Service conditions 
refer to the scope, distribution, and quality of publicly sponsored provisionary and assurance 
endeavors for citizen well-being, such as the provision of public education, public regulation of 
health and financial services, and welfare assistance. Shifts in the quantity and/or quality of these 
resources and services can often prompt policy actors to target them for collaborative efforts, 
where the latter are intended to improve quality or to expand or limit resource access (Emerson 
and Nabatchi 2015; Bryson, Crosby, and Stone 2006; Ostrom 1990). However, the nature and 
specific type of resource or service, as well as its jurisdictional domain, may constrain who is 
motivated to come to the table, who can come to the table, and who even knows the table exists. 


Institutions 


Another factor influencing the decision to collaborate, as well as the trajectory of collaborative 
policy design and implementation, is the institutional framework in which policy actors are 
embedded. Institutions (defined here as formal and informal rules) can either facilitate collabo- 
ration by placing actors in close proximity during the policy process or act as barriers or obstacles 
to collaboration through jurisdictional, informational, or social divisors. Examples of this insti- 
tutional factor bringing potential collaborators into closer proximity include the Administrative 
Procedure Act’s (APA) requirements regarding public participation and stakeholder awareness 
(Bingham, Nabatchi, and O’Leary 2005) and budgetary rules imposed by a bureaucratic agency 
that incentivize stakeholders to collaborate and reach consensus in order to take advantage of 
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available funds (Doberstein 2016). Alternatively, institutional frameworks such as transparency 
requirements can also inhibit collaboration, as Amsler (2016) suggests, by preventing potential 
collaborators from obtaining the confidentiality necessary to work together and/or build trust. 
Rules such as those for agenda-setting and modification can constrain dialogue between indi- 
viduals or groups, thereby potentially inhibiting collaboration (Amsler 2016). 


Political Dimensions and Power Relations 


Collaboration is largely about relationships. The power dynamics introduced by an existing 
system environment into these relationships can, and frequently do, influence considerations 
regarding the need for collaboration as well as the form a collaborative governance arrangement 
takes. Politics can often introduce power relations and potentially power imbalances into the 
collaborative environment, particularly through chains of principal-agent relationships such as 
between elected legislators and citizens (Epstein and O’Halloran 1999; Waterman and Meier 
1998), legislators and bureaucrats (Weingast 1984; Miller 2005), and bureaucrats and non- 
government contractors (Milward and Provan 2000, 2003). However, unequal distributions of 
power can also result from resource conditions such as property rights (Ostrom 2005; Agrawal 
and Ostrom 2001) and resource availability and dependency (Purdy 2012; Casciaro and Piskor- 
ski 2005), as well as population characteristics such as income inequality (Dobbin and Lubell 
2021; Bryson, Crosby, and Seo 2020) and social inclusion or exclusion (Ansell and Gash 2008; 
Newman et al. 2004). 


Social Connectedness, Trust, and Legitimacy 


In addition to the political and/or power dynamics at play, the policy system environment can 
also have pre-existing socially and perceptually based factors encapsulated as actor history, trust, 
and legitimacy that can either catalyze or stymie a collaboration. For example, past coopera- 
tion or conflicts between potential collaborators in the policy environment can either facilitate 
or hinder collaboration through effects on trust and the perceived legitimacy of working with 
other actors (Ansell and Gash 2008; Emerson and Nabatchi 2015). This is important because 
the opportunity to legitimize not only oneself and/or one’s collaboration, but also one’s goal 
or desired state of being (e.g., for a resource/service condition) can give weight to an actor’s 
decision to collaborate (O’Leary and Vij 2012; Chen 2010; Bryson, Crosby, and Stone 2006). 
Moreover, higher degrees of trust can make the decision to work together easier, thereby serving 
as a facilitating factor in the decision to engage in collaborative governance (Huxham et al. 2000; 
Ansell and Gash 2008; Emerson and Nabatchi 2015; Siddiki, Kim, and Leach 2017). 


Problem-centric Drivers 


Finally, decisions to collaborate in policy creation and implementation are not typically straight- 
forward or binary. They are choices about how much to invest in constructing, deploying, or 
engaging in a collaborative governance arrangement as a policy response to a specific problem 
or set of problems bearing particular characteristics. Therefore, the specific characteristics of 
the problem are also important in influencing the form a collaborative governance arrangement 
takes. In their synthesis of public policy and management literature, Scott and Thomas (2017) 
identify several possible paths through which problem identification and understanding translate 
into policy formulation and the form of collaboration adopted. Prominent among these are the 
uncertainty regarding the problem and how to address it, the interdependence among actors 
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affected by and/or responding to the problem, the consequential incentives for potential policy 
collaborators in working together to address the problem, and the ability of leadership to initiate 
and guide the collaborative effort as it is adopted (Emerson, Nabatchi, and Balogh 2012). 


Designing Collaboration as a Policy Tool in the Public Policy Process 


The preceding section highlights factors that contribute to collaborative forms of governance 
and is guided by the general position that collaborative governance is a kind of policy tool 
engaged in the policy process. In this section, we review literature that addresses how col- 
laborative governance is designed through public policies. While collaborative governance can 
take many forms, as further described in the following section, there are specific instances in 
which it is formally codified through public policies. Recognizing them as formal entities of 
governance — that is, as formal kinds of policy tools — policies identify participants in collabora- 
tive governance arrangements, delineate the array of activities that occur among them, structure 
decision-making processes, and essentially lay out the goals of collaborations. 

Much of the research published to date that examines how policies structure collabora- 
tive governance is informed by institutional theories and frameworks, such as the institutional 
analysis and development framework (Ostrom 2005), common-pool resource theory (Ostrom 
1990), and the ecology of games framework (Lubell 2013). In each of these theories and frame- 
works, attention is paid to the way that rules (generally conceived of in terms of “institutions’’) 
structure collective action engaged in some aspect of governance. A central question on which 
this research orients is “How are institutions that govern collaborative venues and processes 
designed?” 

According to Ansell and Gash (2008), who define institutions as the basic protocols and 
ground rules for collaboration, the design of institutions matters insofar as it is through institu- 
tions that conveners of collaboration specify who will be included therein. Other scholars have 
specifically sought to investigate this feature of institutional design (Koski et al. 2018; Siddiki 
et al. 2015). Both Siddiki et al. (2015) and Koski et al. (2018) analyze policy mandates that 
structure collaborative arrangements engaged in food system governance to understand how 
they instruct the composition of stakeholders involved in them. Drawing on this assessment of 
policies and stakeholder composition expressed therein, they seek to understand how mandated 
stakeholder participation materializes in practice, as well as how patterns in stakeholder compo- 
sition associate with the performance of food system collaboratives. 

Beyond stakeholder composition, scholars have also been interested in understanding how 
institutions structure the processes through which participants interact within collaborative gov- 
ernance arrangements. Referencing the treatment of institutions within the ecology of games 
framework, Lubell (2013) notes, “Policy institutions consist of sets of formal and informal norms 
that structure how actors make collective decisions about the ‘operational’ rules governing on- 
the-ground decisions about particular policy issues, such as appropriation of resources and 
provision of public goods” (Ostrom 1990). Institutions are thus recognized for specifying deci- 
sion-making protocols as well as for enabling and constraining actions of participants interacting 
within collaborative arrangements. Olivier (2019) demonstrates how to analyze “networks of 
prescribed interactions” embedded in the design of institutions governing collaborative arrange- 
ments involved in water quality management. His analytical approach allows him to assess how 
responsibilities germane to the collaboration are distributed across participants. Complementing 
Olivier’s work, Herzog, Ingold, and Schlager (2021) empirically compare networks of pre- 
scribed interactions and perceptions of how participants actually interact within collaborative 
arrangements. 
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Finally, beyond defining stakeholders, decision-making processes, and prescribed actions, 
scholars note the important role that institutions play in structuring the goals of collaborative 
governance arrangements. In their study of food system collaboratives, Siddiki et al. (2015) also 
evaluate how policy mandates governing these groups articulate collaboration intent. 

Altogether, the research highlighted in this section describes the various ways that collabora- 
tion is structured through public policies once it is selected as a policy tool with which to address 
a public problem. 


Realizing and Sustaining Collaboration in the Public Policy Process 


Having discussed factors that enable the engagement of collaborative governance as a policy tool 
in the policy process and how policies are used to structure it once it is selected as a policy tool, 
in this section, we turn to discussing how collaboration materializes in the policy process and 
factors contributing to its sustainability and efficacy. Attention to the latter is important because 
while the contours of collaboration are often outlined in public policies — in terms of partici- 
pants, scope of activities, decision-making processes, and goals — the outcomes of collaboration 
are far from determined (Vangen, Hayes, and Cornforth 2015). Rather, studies suggest that even 
when collaborative arrangements are similarly designed, the processes observed within them 
(Bell and Scott 2020) and their results (Scott 2015) vary. 

While collaboration takes many forms and scholars differentiate between them in several 
ways, distinctions are largely based on the time and resource commitments of the participants 
and the extent to which organizations adapt their internal goals and structures to fit within the 
larger whole. In an early study, Mandell and Steelman (2003) differentiate between four forms 
of collaboration, ranging from intermittent coordination, as in the case of disaster response, to 
coalitions and networks in which participants are involved in long-term, interdependent, and 
strategic action. Yet even within network forms of collaboration, there are differences in the 
extent to which participants are interdependent. On the one hand, there are networks with lim- 
ited interdependence — where participants primarily share information — and on the other are 
action networks — where participants are collectively engaged in formally adopted network-level 
courses of action (Mcguire 2006). 

For collaboration to work, managers must address several challenges — some of which are 
understood during the policy design process, others that are not. One set of issues is related 
to the division and allocation of tasks and their coordination (Kenis and Raab 2020). Another 
is related to what Emerson, Nabatchi, and Balogh (2012) refer to as “shared motivation.” To 
implement policy, collaborative public managers must overcome power asymmetries to build 
trust (Ran and Qi 2019; Vermeiren, Raeymaeckers, and Beagles 2021), develop a common 
understanding of how to balance the unity/diversity tension (Saz-Carranza and Ospina 2011; 
Vangen and Winchester 2014), create a shared identity (Mandell, Keast, and Chamberlain 2017), 
and secure ongoing commitment and the contribution of resources from stakeholders (Koliba, 
Zia, and Mills 2011; Lemaire and Provan 2018). 

Scholars have identified several management strategies associated with positive collaborative 
outcomes — both in terms of outputs and the quality of relationships (Provan and Milward 2001; 
Smith 2020; Turrini et al. 2010; Varda, Shoup, and Miller 2012). Increasingly, however, col- 
laborative management scholars are adopting the perspective that few if any of these strategies 
can be understood in isolation. Rather, they must be understood in combination with aspects 
of process, structure, and management practice, along with characteristics of the participants and 
their environment — the idea being that multiple combinations of factors can equally lead to suc- 
cessful collaboration (Ansell et al. 2020; Beagles 2021; Cepiku et al. 2021; Cristofoli, Trivellato, 
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and Verzillo 2019; Markovic 2017; Provan and Kenis 2008; Raab, Lemaire, and Provan 2013; 
Raab, Mannak, and Cambré 2015; Wang 2016). 


Deliberative Forums, Participation, and Process 


Deliberative forums are an important design choice engaged in collaborations to structure par- 
ticipation, cultivate relationships (Raeymaeckers and Kenis 2016; Scott and Thomas 2015), and 
foster principled engagement and shared motivation (Emerson, Nabatchi, and Balogh 2012). 
The contours of these are often specified in policy documents and can range from local com- 
munity working groups to steering committees and the formal governance boards of coalitions 
and networks. 

Although collaboration is considered a nonhierarchical form of organizing, studies find that 
the concept of vertical complexity is useful for understanding the different combinations of 
deliberative forums that exist (Vermeiren, Raeymaeckers, and Beagles 2021). Differences in 
combinations are found to be associated with the task of the collaboration (Beagles 2021; Iborra 
et al. 2018). They are used to balance tensions, such as tensions between inclusiveness and effi- 
ciency or autonomy and accountability (Provan and Kenis 2008; Vangen, Hayes, and Cornforth 
2015), and can help balance the uneven distribution of power and build trust (Vermeiren, Raey- 
maeckers, and Beagles 2021). For example, Iborra et al. (2018) find that within collaborations, 
sanctioning authority is associated with more complex combinations of deliberative forums than 
those focused solely on rule-making. 

While many of these forums are defined in policy documents and are the result of power 
dynamics (Saz-carranza, Iborra, and Albareda 2015), not all forums are prescribed nor must they 
all be activated. Rather, there is considerable opportunity for collaborative managers to activate 
and deactivate these forums as conditions change. For example, in their study of municipal 
family-service networks in Belgium, Vermeiren, Raeymaeckers, and Beagles (2021) find that 
collaborative managers in several collaborations established steering committees and working 
groups at different times in response to issues of mistrust and questioned legitimacy raised by 
participants. And, in their study of a neighborhood regeneration collaboration, Vangen, Hayes, 
and Cornforth (2015) observe how different collaborative forums outlined in the original policy 
design were allowed to go dormant after their usefulness had run its course. 


Social Ties, Integration, and Governance 


Because collaboration is considered a voluntary, nonhierarchical form of organizing, informal 
social structures rather than formal bureaucratic ones are thought to be especially important for 
understanding both the effectiveness of coordination and the development of shared motivation. 
Yet, to a large extent, these social structures cannot be prespecified in policy documents. Rather, 
they must be cultivated. 

In terms of effectively coordinating the work of autonomous participants, studies recognize 
that there is no single best way to integrate collaborative activities. Rather, there are a series of 
more or less complementary goals that must be balanced (Provan and Kenis 2008; Vangen and 
Winchester 2014). And the collective work of autonomous organizations can be effectively 
integrated through both a sparse set of relationships centralized around a single or a few coor- 
dinators (Provan and Milward 1995; Raab, Mannak, and Cambré 2015) or through a dense set 
of relationships connecting a decentralized set of participants (Provan and Milward 1995; Wang 
2016). However, several contextual factors seem to influence whether one or the other form 
of integration is effective — including the size of the collaboration (Beagles 2021; Wang 2016), 
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its resource munificence and stability (Provan and Milward 1995; Raab, Mannak, and Cambré 
2015; Turrini et al. 2010; Wang 2016), and the nature of participants’ mutual understanding. 

Building on this perspective, network governance scholars describe three ideal forms of 
network coordination used within interorganizational collaboration — shared, lead organization, 
and network administrative organization (NAO) — and propose that each form is most effective 
within a distinct set of functional and social conditions (Provan and Kenis 2008). The shared 
form of coordination is the simplest and most decentralized, in which participants coordinate 
among themselves without the use of any centralized administration. 

This form of coordination is thought to be most effective when there are few participants and 
a high density of trust and mutual understanding among them. The lead organization form of 
coordination, on the other hand, is considered the most centralized. It is a form of coordination 
in which one of the active participants in the collaboration takes responsibility not only for its 
own work but also for the activities of the collaboration as a whole. This form of coordination 
is thought to be most effective when there are a moderate number of participants and mutual 
understanding, with most of the trust focused on this central participant rather than among the 
participants themselves. The NAO form of coordination is the most complex. It is a central- 
ized form of coordination. However, unlike a lead organization, the sole purpose of the NAO 
is to coordinate collective activities. It is thought to be able to handle the largest and most 
complex collaborations, and because it is governed by an external entity (Saz-carranza, Iborra, 
and Albareda 2015) or by a board of the participants, the need for dense or a highly centralized 
level of trust is moderated by both the neutrality of the coordinator and the greater potential for 
participatory group decision-making. 


Collaborative Management and Orchestration 


Regardless of the deliberative forums and coordination structure, an important role of collab- 
orative management is building and maintaining trusting relationships and developing a shared 
motivation among collaborative participants more than formal oversight (Bartelings et al. 2017; 
McGuire 2006; Piatak et al. 2018). With this in mind, scholars have increasingly focused on the 
different types of relationships collaborative participants form, their effects, and the role collab- 
orative managers have in developing and maintaining them. For example, studies find that rather 
than dense networks of asymmetrical relationships, sparse networks with a significant number of 
clustered, reciprocal relationships are associated with effective collaborations (Provan and Sebas- 
tian 1998; Ulibarri and Scott 2017; Yi 2017). Other studies find that these clusters of strong ties, 
especially when they include the central coordinator or other champions of the collaboration, 
improve goal congruence and the willingness of participants to contribute resources and effort 
(Lemaire 2020; Lemaire and Provan 2018). 

However, again, the types of relationships and the role of the collaborative manager in these 
relationships seem to vary depending on the context. Paquin and Howard-Grenville (2013) find 
that a collaborative manager’s orchestration role varies over time, depending on their resources 
and the needs of participants. In the case of the National Industrial Symbiosis Programme 
(NISP), the collaborative manager shifted roles from “encouraging serendipitous encounters 
between network members (‘blind dates’) to increasingly selecting members and more closely 
influencing their interactions (‘arranging marriages’)” (p. 163). Expanding on these findings, 
Cristofoli and colleagues (2019) find that while in young, densely connected networks, collab- 
orative managers are perceived to be effective when focusing on stabilizing activities like build- 
ing trust among participants, in more mature collaborations, effective managers move beyond a 
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stabilizing role to an orchestrating role and then to a networking role where new members are 
brought in — presumably based on the functional needs of the collaboration (Ansell et al. 2020). 
Other studies suggest that the culture developed within the collaboration is also important for 
understanding the types of relationships that are formed and found to be effective (Markovic 
2017). 


Policy Evaluation, Feedback, and Learning 


Finally, in an age where performance management, impact, learning, and accountability are 
dominant values, collaboration is seen by many not only as a tool that supports attainment of 
particular policy goals, but also for (re)building social capital within a community. Because of 
this, there is interest in collaborative policy implementation as well as collaborative policy evalu- 
ation, feedback, and learning (Gray, Jenkins, and Leeuw 2017). Didham and Ofei-Manu (2020) 
provide a positive example of such when they describe how, together, cooperative inquiry, dia- 
logue and deliberation, and knowledge co-production served to improve the monitoring and 
evaluation of an education for sustainable development program across seven countries: more 
specifically, how the process resulted in the perception that the system fit the needs of both 
national and international policymakers and the local educational districts. 

Grounded in this line of research, evaluation scholars describe three broad functions of 
collaborative inquiry — accountability, learning, and transformation — and describe a set of 
evidence-based principles to guide choices in collaborative approaches (Cousins et al. 2020; 
Shulha et al. 2016). Differences in these approaches are described based on how they balance 
considerations, like control of technical decision-making, diversity of stakeholders involved in 
collaboration, and the depth of participation among stakeholders (Cousins et al. 2020; Vangen 
and Winchester 2014), with recent studies observing that some factors designed to promote 
intra-agency performance evaluation can actually undermine inter-agency collaboration (Choi 
and Moynihan 2019). 

As a strategy for addressing these different performance management and learning needs 
and the tensions they create, Douglas and Ansell (2021) describe three general forms of perfor- 
mance regimes used in collaborations — each balancing considerations of collaborative inquiry 
differently — and argue for the creation of regular summits designed specifically for the needs and 
priorities of the collaboration, intentionally balancing the need for shared understanding with 
requests for commitment, adjustment, and resources. 


Conclusion 


This chapter presents collaborative governance as a kind of policy tool engaged in the policy 
process. The focus on collaboration reflects its ubiquity in scholarship and practice as a key 
mechanism through which governments pursue their work. The chapter identifies factors 
that contribute to the engagement of collaborative governance in policymaking, ways it is 
structured through public policy, and ways that collaborative governance manifests follow- 
ing its engagement and design, as well as factors contributing to sustained collaboration and 
collaboration efficacy. As policy scholars continue to study collaborative governance in the 
policy process, continued characterization of it as a policy tool may help situate collaborative 
governance within a more expansive array of policy literature. This, in turn, may facilitate 
the development of novel questions and insights about the antecedents, design, and efficacy 
of collaborative governance. 
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SUPPORTING DECISION-MAKING 
WITHIN THE POLICY CYCLE 


Techniques and Tools 


Irene Pluchinotta, Katherine A. Daniell and Alexis Tsoukias 


The policy cycle is a model of interactive stages for structuring policy problems and making 
decisions to deal with them. The model encourages cycling through discrete activities, using dif- 
ferent policy tools. In order to support decision-making within a policy cycle, a decision-aiding 
perspective is needed: namely, formal tools and methods introduced to help decision-makers 
develop reasoned decisions during the different phases of the policy-making process. 

The intention of this chapter is firstly, to introduce existing decision analysis analytic methods 
and tools that can be used to support the different phases of the policy cycle, using concepts from 
decision analysis and policy analytics. Secondly, the decision-aiding perspective for the design 
and evaluation of policies is introduced and discussed. Lastly, the chapter focuses on the impor- 
tance of the policy design phase, introducing an innovative participatory tool to support policy 
makers and relevant stakeholders during the design of policy alternatives 


Introduction 


The process of designing, implementing and assessing public policies is a major challenge for 
policymakers (De Marchi, Lucertini, and Tsoukias 2016), and having strong knowledge of a 
range of policy tools that could be deployed is a prerequisite for achieving the policy goals 
(Howlett 2018). Policymaking involves the deliberate and conscious attempt to define policy 
goals and connect them to instruments or tools expected to realise those objectives (Howlett, 
Mukherjee, and Woo 2015). 

Policymaking is a non-linear process and has been conceptualised in a myriad of ways that 
express this complexity. Some models have great explanatory power for understanding the 
interrelationships of politics, values, issues and decisions (e.g., Lindblom and Woodhouse 1968; 
Cohen, March, and Olsen 1972; Kingdon 1984; Dunn 1981; Sabatier 2007); however, they 
have proven less helpful for understanding where decision support tools can help policymakers. 

Other models are criticised for oversimplification of processes (e.g., Jann and Wegrich 2007 
on Lasswell’s policy cycle) but have stood the test of time as practice-focused aids, particularly for 
public servants supporting a range of policy activities. Indeed, the cyclical approach introduced 
by the policy cycle (Lasswell 1956) enables intricate phenomenon of policymaking to be disag- 
gregated into manageable steps, allowing policy analysts to focus on the different issues of each 
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phase (Bridgman and Davis 1998) using specific tools. Such models of the policy cycle have 
been used in many countries, including Australia, for decades. 

Policy decisions are often supported by incorporating appropriate literature, traditional forms 
of policy analysis and key lessons from policy-making studies at the relevant steps of the policy 
cycle. However, to support decision-making within the policy cycle, a decision-aiding perspec- 
tive is needed (De Marchi, Lucertini, and Tsoukids 2016): namely, formal tools and methods 
introduced to help policymakers improve their decisions during the different phases of the 
policy-making process. A decision-aiding process supports the construction of artefacts using 
formal and abstract languages, with shared and consensual rationality models (see Tsoukias 2007). 

For instance, few government departments and public organisations have yet managed to 
make systematic use of the broad range of data, evidence, decision analysis, operational research 
methods, cutting-edge statistical, machine-learning modeling techniques and also participatory 
approaches to inform their work (Daniell, Morton, and Rios Insua 2016), even though data 
analytics groups are being set up in governments across the world to start to wrangle what gov- 
ernment departments and partners are collecting and storing. 

Within this context, the concept of “policy analytics” was introduced in 2013 as an attempt 
to develop a framework, tools and methods to address the challenges related to policymaking 
(De Marchi, Lucertini, and Tsoukias 2016). 

Policy analytics is defined by Tsoukias et al. (2013) as a project to 


support policy-makers in a way that is meaningful (in a sense of being relevant and add- 
ing value to the process), operational (in a sense of being practically feasible) and legiti- 
mating (in the sense of ensuring transparency and accountability), [by drawing] on a 
wide range of existing data and knowledge (including scientific knowledge, and expert 
knowledge in its many forms) and [combining] this with a constructive approach to 
surfacing, modelling and understanding the opinions, values and judgments of the 
range of relevant stakeholders. 


This conceptual initiative prompted numerous research teams to develop empirical applications 
of this framework and to reflect on their own decision support practice at the science-policy 
interface (see Meinard et al. 2021). 

Furthermore, as discussed by Ferretti, Pluchinotta, and Tsoukias (2019), when it comes to 
policymaking, decision analysis has developed, among others, methods which aim at supporting 
different phases of the policy cycle (Larson and Odoni 1981; Larson 2002; Sinuany-Stern and 
Sherman 2014). However, most emphasis in this stream has been on problem structuring and 
on the rational selection among given alternatives, resulting in the development of guidelines 
for public policy evaluation at different levels (e.g., the Green and Magenta Books of the UK 
government, the Public Policy Assessment Book of the UK government, the European Social 
Fund Manual) but with limited consideration to support the specific policy design phase. 

The aim of this chapter is to introduce existing decision analysis analytic methods and tools 
that can be used to support the different phase of the policy cycle, using concepts from decision 
analysis and policy analytics. Secondly, the decision-aiding perspective for the design and evalu- 
ation of policies is introduced and discussed. Lastly, the chapter focuses on the importance of the 
policy design phase, introducing an innovative participatory tool to support policymakers and 
relevant stakeholders during the design of policy alternatives. 

The remainder of the chapter is organized as follows: the decision-aiding perspective is dis- 
cussed in section 2, the policy cycle analytics tools are presented in section 3, while section 4 
introduces a tool for the generation of policy alternatives. Section 5 concludes this chapter. 
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A Decision-Aiding Perspective 


From a decision-aiding perspective, what we can observe and model is the existence of nested 
decision-aiding processes within the policy cycle. We can distinguish at least two levels. The 
first one is the policy cycle itself, considering any stakeholder involved in the policy cycle ask- 
ing some advice on how to conduct themselves within it. This will generate a decision-aiding 
process of a more strategic nature on how to structure the inter-organisational decision pro- 
cess characterising the policy-making process (Ostanello and Tsoukias 1993; Chisholm 1972; 
Holmqvist 2003; Majchrzak, Jarvenpaa, and Bagherzadeh 2015). The second level concerns 
each single phase of the policy cycle. For any of these, any stakeholder involved in the policy 
cycle might request the support of a team of analysts, these using formal decision support tools. 

A more formal definition of what a decision-aiding process 1s has been introduced in Tsoukias 
(2007), where the process is characterised through four cognitive artefacts (or deliverables) of 
the process: (1) a representation of the problem situation (stakeholders, stakes, resources), (2) a 
problem formulation (actions, attributes, problem statement), (3) an evaluation model (alterna- 
tives, measures, preferences, protocols and algorithms) and (4) a final recommendation. Mazri, 
Tsoukias, and Daniell (2019) extended this definition to the case of participative decision pro- 
cesses (more relevant in the policy-making context) since defining the participation structure 
is, on its turn, a decision problem, once again to be supported through formal decision support 
tools. 

Colorni and Tsoukias (2013, 2018, 2020) introduced a formal structure of what a decision 
problem is (a problem formulation) and focused on the generation of actions and alternatives. 
There are two findings which are relevant for the purpose of this chapter. The first is the fact 
that designing the alternatives of a decision problem is a decision problem itself: this establishes a 
recursion of decision problems, extending the formal structure of decision aiding processes. The 
second is the fact that creating the recursion of decision problems can be modeled using design 
theory concepts (LeMasson et al. 2018). 

Designing the set of decision options (which get the values of the client) is a neglected topic 
although as important as evaluating and choosing them (Ferretti, Pluchinotta, and Tsoukias 
2019). This new perspective allows considering integrating formal design support tools within 
formal decision support tools (and vice versa), a topic yet to be developed in the specialised 
literature. 


Policy Cycle Analytics Tools 


The policy cycle approach suggests that policy develops through a set of interrelated tasks for 
structuring policy problems and making choices concerning them, from setting the agenda 
and developing policy options to policy evaluation and revision (Daniell, Morton, and Rios 
Insua 2016). The model encourages cycling through discrete activities, using different policy 
tools. Adapted from Althaus, Bridgman, and Davis (1998) and Daniell, Morton, and Rios Insua 
(2016), the version of the policy cycle used for this chapter includes: 


Agenda-setting and issues identification to define the problems of public concern 
that require policy action or changing and establishing priorities. 

Policy formulation to reach a better understanding of a public issue on the agenda 
and the define the boundaries of the policy goals and objectives. 

Policy design to create policy alternatives, representing possible paths for the solu- 
tion of a policy problem connecting the initial state of the problem (i.e., the undesirable 
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state) to the final state. The problem is formulated in the previous phase, and alterna- 
tive policy options are developed. 

Policy analysis and decision for the selection of the preferred policy option. Based 
on the analysis of the different policy alternatives, a final decision is made, and the 
chosen policy proposal is fully specified. 

Policy consultation to test the policy proposal and to gather support for a con- 
structive policy initiative. This phase involves more than one agency and/or non- 
governmental stakeholder for the improvement and test of the policy and, when 
appropriate, for gathering support. 

Policy implementation to pursuit the goal agreed on in the first phase and to put the 
policy into practice. At this stage, the necessary public resources and regulations are 
mobilised to make the policy operational. 

Policy monitoring and evaluation to observe and assess, on an ongoing basis, whether 
the implemented policy is producing the expected results and to identify whether the 
policy should be changed or new issues need to be considered in the agenda. 

Policy learning and readjustment to rethink the policy as appropriate after a recon- 
sideration of the implemented instruments and related policy outcomes. 


We provide an overview of key tools from the decision analysis research community to intro- 
duce analytical methods commonly employed in each phase of the policy cycle. Bridging the 
two research fields (1.e., operational research and policy science), we aim to introduce a number 
of operational research tools, opening a multidisciplinary dialogue and underlining the need 
for a systematic mapping of existing tools. Potentially, the decision-aiding perspective could be 
applied in each phase of the policy cycle; however, there are areas with no or limited operational 
research applications. It is worth clarifying that we do not claim to provide a comprehensive list 
of all the existing operational research methods that have been applied to one or more phases 
of the policy cycle. 

Using the eight-phase policy cycle, we included two types of tools — analyst-driven analytics 
and stakeholder-driven analytics — since each phase could be carried out via public consultation 
or, more generally, using collaborative and participatory approaches. As highlighted by Pluchi- 
notta et al. (2019), public participation is widely documented as being a valuable component 
of policymaking (Beierle and Cayford 2002; Beierle 2002), bringing the need for facilitating 
stakeholders’ contributions (Ackermann and Eden 2011). 

Figure 19.1 summarizes the existing methods and tools that can be used to support the dif- 
ferent phases of the policy cycle, using concepts from decision analysis and policy analytics, 
showing the lack of formal tools for supporting decisions in all the phases of the policy cycle. 

There are few phases with a large number of decision analysis methodologies successfully 
adapted and integrated to support participatory policy-making processes. For instance, both 
phases agenda setting and policy formulation have: soft operational research approaches (Checkland 
2000; Pollock, Rothkopf, and Barnett 1994); problem structuring methods (Rosenhead 1989), 
such as strategic options development and analysis (Eden 2004) and boundary critique (Midgley 
and Pinzon 2011); and behavioural operational research (Hämäläinen, Luoma, and Saarinen 
2013; Franco et al. 2021). Furthermore, system thinking (Sterman 2000) and group model 
building (Vennix 1996) are often used to carry out policy, governmental and societal system 
analysis. 

In relation to the phase policy analysis and decision, several decision analysis approaches exists, 
including both analyst- and stakeholder-driven analytics, such as the strategic choice approach 
(Friend and Hickling 1987), system dynamics (Sterman 2000) and participatory modeling 
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Figure 19.1 Policy Cycle Analytics Tools 


(Acronym: operational research [OR]) 


IPAO Aayod ays urynyy Suryoyy-uotsi9aqq 


Irene Pluchinotta et al. 


(Voinov and Bousquet 2010; Voinov et al. 2016), with simulation and scenario analysis, behav- 
ioural operational research, data envelopment analysis and performance measurement (Charnes, 
Cooper, and Rhodes 1979; Cook and Seiford 2009; Emrouznejad and Barnett 2007), cost- 
benefit analysis and multi-criteria decision analysis (Belton and Stewart 2002). 

On the other side, algorithmic models can be used to contribute to evidence bases and are 
increasingly used to inform policymakers (Kolkman 2020). There are two aspects of digital 
transformation we may cite when we talk about policy analytics tools. The first concerns the 
active systemic use of data in different phases of the policy cycle (e.g., for issue identification, 
policy analysis, evaluation [pay-for-performance approach for energy efficiency programs to 
evaluate energy efficiency program success]) and for designing data-dependent policies (e.g., 
personalised taxes based on consumption patterns) (Janssen and Kuk 2016). The second relates 
to the passive use of algorithms in running utilities and services, introducing the concept of a 
policy designed by induction. For instance, methods such as deep reinforcement learning are 
increasingly being used to find optimal policies for a given control task (e.g., Dhebar et al. 2020; 
Behzadan and Munir 2017). 


Policy Design: A Tool for the Generation of Alternatives 


Within the policy cycle, policy design can be defined as a specific form of knowledge gathering 
about possible policy alternatives on policy targets and the application of that knowledge to the 
development of policies aimed at the attainment of specifically desired public policy outcomes 
and ambitions (e.g., Bobrow 2006; Capano and Howlett 2015). 

Therefore, policy design is an intricate challenge for policymakers as future policy outcomes 
are inherently uncertain (Nair and Howlett 2016). Evidence suggests that policies often fail due 
to the lack of formalised policy design methodologies for the innovative generation of alterna- 
tives (Howlett 2014). Policy design has long been seen as a component of policy development 
without any operational characteristics (Howlett 2011). Although policy design represents a 
crucial step since it determines the quality of the alternative policies being considered, the for- 
malisation of this process has been little investigated (e.g., Ferretti, Pluchinotta, and Tsoukias 
2019). 

Using the lens of a decision-aiding perspective, policy design can be considered the result 
of a collective decision-making process involving multiple stakeholders for the generation of a 
set of policy alternatives (Pluchinotta et al. 2019). Alternatives tend to be few and similar when 
the policy design process is constrained (Alexander 1982). In contrast, a decision-aiding process 
can bring novelty through the expansion of the set of solutions (see Colorni and Tsoukias 2020, 
2018). 

Ferretti, Pluchinotta, and Tsoukias (2019) argue that the mainstream decision analysis litera- 
ture focuses on tools to support a rational selection among given alternatives. Despite this, policy 
design represents a crucial step since it determines the quality of the alternative policies being 
considered; the formalisation of this process has been little investigated. 

Within this context, Pluchinotta et al. (2019) underline that there is a demand for meth- 
odologies to support policymakers and relevant stakeholders during the design of policy alter- 
natives, exploring the operational role of design theory, and specifically concept-knowledge 
(C-K) theory, to develop a formal tool for the generation of policy alternatives. The tool is 
based on the integration between problem structuring methods (PSMs) and C-K theory. PSMs 
are implemented to elicit and structure individual problem understandings and to detect and 
analyse differences among different stakeholders’ concerns and values (Mingers and Rosenhead 
2004). C-K theory framework is then meant to facilitate the alignment of the different problem 
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frames and available knowledge and to enable the creative process for developing innovative and 
consensual policy alternatives. 

Briefly, P-KCP consists of three main phases: the K-space phase aims to identify a shared 
concern, to gather missing information and to build a comprehensive summary of the knowl- 
edge about the issue under consideration by combining individual stakeholders’ knowledge in 
order to support the subsequent generative phase (C-space phase). Afterwards, the C-space is 
built during a one-day stakeholder workshop: stakeholders identify and discuss the traditional 
policy alternatives and propose innovative ones through the expansion of the C-space. Follow- 
ing the C-K theory framework, the C-space represents the map of all identified possibilities, 
improving the search for new alternatives. Lastly, the project phase uses the K- and C-spaces to 
shape the policy recommendations. 

C-K theory’s underlining idea is that design is a generative process through which something 
unknown can intentionally emerge from what is known (Hatchuel 2001). Briefly, C-K theory 
is based on the distinction between two expandable spaces: a space of concepts (C-space) and 
a space of knowledge (K-space) (LeMasson, Dorst, and Subrahmanian 2013). The process of 
design is therefore defined as the co-evolution of C- and K-spaces through four types of inde- 
pendent operators (C-C, C-K, K-K, K-C): namely, concepts generate other concepts or are 
transformed into knowledge, and knowledge generates more knowledge or helps formulating 
concepts (Hatchuel and Weil 2002, 2003). According to Hatchuel and Weil (2009), the K-space 
is a space of propositions that have a logical status (i.e., “true” or “false’’) for a designer in a given 
time step. Whereas the C-space is a set of propositions describing an object (e.g., our policy), 
that has no logical status in the current K-space (i.e., when a concept is formulated, it is impos- 
sible to prove that it is a proposition of the K-space). 

Over the last few years, C-K theory has gained growing academic and industrial interest 
(Agogué and Kazakci 2014). C-K is a theory of reasoning for innovative design situations, 
overcoming the limits of traditional design theory (Hatchuel et al. 2015) and creativity methods 
(Kazakci and Tsoukias 2005). It provides researchers and practitioners with a framework to 
describe and analyse innovative design processes for the generation of alternatives (Pluchinotta 
et al. 2020). For a detailed discussion on a C-K theory—based tool for policy design — namely, 
policy-KCP (P-KCP), see Pluchinotta et al. (2019). 


Conclusions 


In this chapter, we adopt a general policy cycle approach to represent the complex set of activities 
occurring when a policy is established: from perceiving the need of a policy to the monitoring 
and feedback analysis of a policy implementation, through the different steps of agenda-setting, 
policy design, choosing policy tools, etc. 

We show that, despite decision analysis tools having been often suggested and actually used 
for supporting the activities within the policy cycle, we lack a comprehensive methodology on 
how to support the whole cycle and several among the steps characterising it. We nevertheless 
provide a short survey of methods and models created for such purposes. Under such a perspec- 
tive, we present in more details how policy design can benefit from formal decision analysis tools 
as well as from formal design theory and, more precisely, C-K theory. 
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Introduction: Looking Ahead in Policy Formulation 


Looking ahead and anticipating conditions is an important aspect of policy formulation (Van 
der Steen et al. 2013, 2010, 2011; Slaughter 2005, 2007; Glenn and Gordon 2006). Looking 
ahead helps policymakers foresee future problems, identify new or changing trends that shape 
the conditions for future policies and see new types of solutions. The ability to anticipate the 
future is a crucial element of solid policy making. 

Public organizations at all levels of government conduct anticipatory studies to be better pre- 
pared for unknown futures. But the practice of applying foresight to policy is often problematic: 
foresight is hard enough to do, but it seems even harder to link futures studies to policy formula- 
tion. It is one thing to do futures studies; it is another thing to do something with them. The 
challenge for futures studies in policy formulation is to execute the techniques and methods for 
anticipation and to link them to the reality and turbulence of policy formulation in the context 
of today. 

This problem is not new. Empirical research into the practice of foresight in policy 
identifies a persistent gap between generating insights about the future and using these 
insights in policy formulation. The problem is not a lack of interest in the future but a mis- 
match between the properties of anticipatory knowledge and policymaking (Riedy 2009; 
Slaughter 2007; Schwartz 1996; Hart and Tindall 2009; Van Asselt et al. 2007; Groves and 
Lempert 2007; Brown et al. 2000; Paillard 2006; Sardar 1999; Cairns et al. 2006; Van der 
Duin et al. 2009; Van der Duin et al. 2010; Van der Duin and Van der Steen 2012). Despite 
examples of successful attempts to link foresight, the tension between present-day interests 
and the long-term dimension of policymaking remains (e.g., see Ho 2012; DaCosta et al. 
2008; Nehme et al. 2012; Konnola et al. 2012; Yoda 2011; Chermack 2004; Colson and 
Corm 2006; Habegger 2009; Groonbridge 2006; Andersson 2010; Chan and Daim 2012; 
Van Asselt et al. 2010; Van der Steen et al. 2013; Noordegraaf et al. 2014; Veenman 2013; 
Kuosa 2011; Fuerth 2009; Van der Duin et al. 2009; Riedy 2009). That is, in this chapter 
we discuss tools for studying the future but also the dilemmas of applying these tools in the 
process of policymaking. 
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Why Systematically Studying the Future Is Helpful for Policymaking 


Futures studies draw attention to the role of time in policy. There are many different ways to 
think about time in relation to policy, but in this chapter, we focus on the role of time as a 
space in which the dynamics of policy take place. Problems, solutions and conditions for policy 
change with time: partly as a result of policy itself, but also by means of other inherent dynamics. 
Conditions are not fixed but change over time in complex cycles of interaction. These interac- 
tions happen in time, at varying pace and intensity. Time is the space in which these chains or 
cycles of interactions take place. Moreover, time is also a space for emergence to take place, 
where ‘new things’ can happen that are currently not yet a part of the system. Social systems are 
not bounded by what is already inside. Actors can innovate, choose new routes, come up with 
fresh answers and express behaviours that were previously not in the system. Time is the space 
in which such emergence occurs. To make policy is to successfully steer the dynamics that take 
place in time. 

Looking ahead can help policymakers not simply project today’s interactions and patterns 
onto a future time frame. In a systematic study of the future, policymakers explore the flow of 
interactions in time. Some of these interactions take little time, with perhaps very radical effects 
in the short term; other interactions take longer, either to occur or for the effects to become 
noticeable. Exploring the future and looking ahead into the space of time means attempting to 
foresee at least some of these interactions and/or the possible conditions and outcomes of these 
interactions. 

Studying these interactions requires a balance between the limitations and possibilities of 
the future. The future is inherently open, in the sense that complex systems can take any possible 
shape. The future is not bounded to the present state of the system. At the same time, the future 
is also embedded in the current state. The present provides the conditions from which the future 
develops: for instance, in demographics and natural conditions but also in social constructions 
that root behaviour. Van Asselt et al. (2010) and Veenman (2013) speak of the future as open, not 
empty. The future can be anything, including completely discontinuous with the past but also 
a mere continuation of the present. That is why in futures studies the future is usually discussed 
in plural: futures-, instead of the future. 

Studying the future is relevant for policymaking in three ways: (1) to identify possible, plau- 
sible and preferable problems; (2) to identify possible, plausible and preferable solutions; and (3) 
to imagine possible, plausible and preferable future conditions. 

Firstly, a temporal perspective calls for a critical analysis of policy problems. Is a problem of 
today still a likely problem in the future? Some present-day problems may disappear or change 
dramatically. Issues that are hardly conceived as problems today could very well constitute the 
policy agenda of the future. And even if the labels of the issue remain the same, a problem can 
be very different in future times; today’s unemployment issue is different from that of the 1960s, 
and in 2030, the problem will likely be different from that of today. 

Secondly, the range of solutions policymakers consider is also time sensitive. Do policymakers 
consider options beyond the present ones and look at solutions from the perspective of possible 
futures dynamics? New technologies facilitate the formation of bottom-up and ad hoc local net- 
works, which generate new solutions. For other issues, the solutions that work well today may 
be ineffective in the future. New solutions may arise, and present-day solutions might become 
obsolete. 

Thirdly, the working conditions for problems and solutions will differ over time. For example, 
society is becoming more diverse, with a variety of cultural and social values and norms. Tech- 
nological innovation could lead to more inequality but also to a more balanced society. What 
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are the professions of the future, and what professions will become obsolete with technological 
development? All these are relevant possible future conditions. Taking the possible and plausible 
future state of these conditions into account can improve the quality of policy formulation. 


How to Look Ahead in Policy Processes 


Two Approaches to Systematically Study the Future: 
Forecasting and Foresight 


In spite of its inherent uncertainty, it is still possible and productive to attempt to ‘look ahead 
in time’ and assess the possible dynamics of a system in future times. Theory about long-term 
analysis typically identifies two main approaches to study possible futures and portray future 
dynamics of a system in the space of time: forecasting and foresight (Bell 2003; Slaughter 1995; 
van der Steen 2017). We will discuss each of them here and also discuss some of the more recent 
developments in both fields (see Table 20.1). 

Forecasting is the tradition in which quantitative projection is used to identify different pos- 
sible futures (Armstrong 2001; van der Steen 2017). A projection is based on a causal model, 
often embedded in current trends and historical data. A crucial choice in forecasting is the 
design of the model: which actors and factors, and which relations, are included in the model 
and are considered — literally — when making the forecast. A forecast ‘calculates’ the outcomes 
of interactions between actors and factors in the model over a certain space of time; it portrays 
paths through the modeled system within the relations within the model and, therefore, also 
within the margins of the model. A forecast is not necessarily a continuation of the status quo, 
but it does inevitably produce a continuation of the ‘reality in the model’; it is continuation 
within the bounds of the model (Walker et al. 2012). 

Foresight is the tradition in which qualitative data is used to initiate debates and thinking 
about possible, plausible and preferable futures (Bell 2003; Van der Duin et al. 2009, 2010; 
Chermack 2004; Wilkinson and Kupers 2014; Klooster and van Asselt 2006; Glenn and Gordon 
2006). There is discussion about patterns, and there is causality involved, but the range of pos- 
sible futures is not limited to a fixed and predefined causal model; causality does not have to be 
precise and final beforehand, but notions about the relevant actors, factors, causes, effects and 
relations can develop during the process of discussing possible futures (WRR 2010). 

Foresight is based on a more qualitative and less strict definition of a model and sees modeling 
more as a noun than a verb. Foresight allows for less linear thinking about potential futures and 
can produce more radical futures; possible relations and discontinuity can be discursively added 


Table 20.1 Forecasting Versus Foresight 


Forecasting Foresight 
Aim To describe how the future will be To describe how the future might be 
Activity Predicting characteristics of the Exploring possible futures 
future 
Product Saying with certainty and probability Identifying conceivable possibilities and ‘key 
uncertainties’ 
Method Using verifiable sources, ‘crisp’ data Based on wider concepts of knowledge, skills and 
and ‘tight’ causal models experiences; ‘fuzzy’ information; a wide range of 


creative, interactive and associated methods 
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to the discussion of possible dynamics and can be considered in imagining possible futures; even 
though the exact nature or ‘value’ of the development is unknown and undefined, it can still 
become part of the discussion of futures dynamics (van der Steen 2017; Paillard 2006; Yoda 
2011; Kuosa 2011; Noordegraaf et al. 2014). 

Forecasting results in statements about how things will be in the future — not in the sense that 
one future is pinpointed as ‘the future’, but the model inherently produces ‘hard’ numbers about 
the range of simulated scenarios; it produces probabilities, effects, costs and benefits. Foresight 
results in images about how things might be, not in terms of numbers, but in narratives, images 
or a ‘look and feel’ of possible futures and futures dynamics (Bell 2003; Burt and van der Heijden 
2003; Van der Heijden 2005). Foresight is about identifying conceivable possibilities — “possible, 
plausible and preferable futures’ — using a broad arsenal of sources that are accepted as both “pos- 
sible’ and ‘workable’ (Ogilvy 2002). 

Because of the central role of the model as the causal environment in which different sce- 
narios are simulated, forecasts are primarily based on quantitative resources. The model requires 
quantitative data and also is based on numerical values for relations: e.g., in order to forecast 
the possible effects of better pensions on the lifestyle of pensioners, the forecaster needs to 
assign a mathematical value to the relation between ‘available income’ and ‘types of behavior’; 
the relation needs to be defined in terms of cause and effect, but also in terms of ‘size’ of the 
relation (e.g., ‘an increase in income of 1.0 results in an increase of 1.2 in time spent on lei- 
sure activities by pensioners’). The relation cannot simply be named; it needs to be numbered 
as well. 

Foresight allows more space for interpretation, intuition and argumentation and looks for 
‘signs’ of change and trends that have yet to be scientifically validated (Splint and van Wijck 
2012; Van der Steen et al. 2011; Maier et al. 2016). Foresight seeks to interpret clues, rather 
than running established facts through a fixed causal model that reflects historic processes and 
causalities (which may not apply to the future in any case), often with several variations in either 
the model or the facts so that diversity in the findings is established (the forecasters’ “scenarios’). 

In a foresight study, desirability and other normative judgments are necessary parts of the 
process. A foresight study does not make or prescribe choices, but it does incorporate notions of 
‘value’ into the scenarios (Slaughter 1995). In the same example of the behaviour of pensioners, 
a foresight study could identify the possible relation between different elements in a system and 
‘name’ different outcomes and second-order or third-order effects of such outcomes; these can 
be considered by introducing them into the discourse about the future and reflect on possible 
outcomes of the relation. 

In a foresight study, it is not necessary to ‘number’ each relation; naming them and bringing 
them into the debate about the future is enough to discursively add them into the narrative of 
possible futures. 


Tools and Techniques for Studying the Future 


There are a wide variety of tools and methods to investigate the future and look for possible, 
plausible and preferable futures (Voros 2003). The choice of tool or technique is important for 
the outcomes of a study, and knowing the variety of options and mastering at least several of 
them are important for anticipatory work. However, there is more to anticipation than merely 
tools; Bell argues that foresight is not primarily about following a methodology but mostly 
about the underlying skills, creativity, analytical capacity and future-oriented mind-set of those 
engaged in the foresight work: ‘No methodology has a monopoly on producing good or bad 
work’ (2003, p. 241). Methods and techniques count but do not guarantee a successful foresight 
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project. Also, not every method appeals equally to the policymakers who need to work with the 
results of a study. Bell stresses that the choice of methods depends on a goodness of fit with the 
skills of the actors involved, the organization, the policy problem at hand and other contextual 
factors. There is a codified methodology, but the choice of a certain method largely depends on 
context. We will briefly discuss a set of anticipatory methods. 


Scenario Studies 


The scenario method (e.g., Van der Heijden 2005; Schwartz 1996; Bell 2003; Van Asselt 2010; 
Van Notten 2005; Van ’t Klooster and Van Asselt et al. 2006) is one of the most commonly used 
anticipatory methods. Scenarios are ‘stories describing different but equally plausible futures 
that are developed using methods that systematically gather perceptions about certainties and 
uncertainties’ (Selin 2006, p. 1). The fundamental assumption of the method is that there are 
deep uncertainties about the future, that many different directions of development are possible, 
but that a coherent method can help explore that inherent variety. The goal is not to predict the 
future but to map the variety and to see the multiplicity of futures. 

The first step in a scenario study is to analyse a wide range of possible developments that may 
affect the future. After that, the scenario team selects the two most powerful forces for the future 
of a certain field, which then become the ‘driving forces’ for the scenarios (Van ’t Klooster and 
Van Asselt 2006; Van der Heijden 2005). The driving forces are put on two scenario axes and 
together constitute four dominant scenarios. The scenarios are then further filled in with more 
or less detail. The purpose of building a variety of scenarios is not to pick and choose the ‘best’ 
scenario but to be prepared for all possible options. Policymakers can aim for a particular future, 
but the true purpose of scenario study is to be prepared for many different possible futures and 
to be able to recognize them early on and adapt when necessary. 

Both the traditions of forecasting and foresight use scenarios. In the tradition of forecasting, 
the scenarios are based on analysis of current trends, which are translated into elaborate models 
that project different possible variants of futures. This is done with quantitative methods that 
greatly benefit from the increased capacity of computers to calculate possible options. That has 
allowed builders of models to produce an enormous variety of options and incorporate highly 
complex relations and many variables into their models. A scenario study from the forecasting 
tradition typically produces scenarios that consist of numbers that represent characteristics of that 
scenario: for example, one scenario of 1% economic growth and one of 3%, with the different 
consequences of each scenario for unemployment, social welfare, the annual budget, national 
debt and so on. 

Scenario studies are used differently in the school of foresight. The point is still to produce 
a variety of possible futures, but the process is more qualitative and involves interaction with 
many stakeholders, creative techniques and large group sessions (e.g., Janoff and Weisbord 2006; 
Novaky 2006; Inayatullah 2005). The scenario team organises interactive sessions with many 
stakeholders to discuss possible trends. Then, there is a discussion to condense ‘driving forces’ 
from the variety of trends. The group (team and participants) construct a matrix that scores the 
level of impact and the level of uncertainty of trends and developments. The two trends that 
have the highest combination and uncertainty become the driving forces of the scenario study 
because they represent the most relevant uncertainties to explore for the purpose of this study. 

Defining the driving forces is a crucial step because the driving forces become the backbone 
for the scenario study and, in this approach, are the product of complex group interactions. But 
it can be risky because groups sometimes produce creative driving forces that ‘feel right’ to the 
group and reflect the consensus in the room but are not considered relevant by experts in the 
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field. That is not to say that the group is wrong or not very good at formulating driving forces; 
experts can be locked in the path of a scientific field and look at the future in the rear-view mir- 
ror; they are bounded by current empirical data, and it is not directly their professional expertise 
to look beyond the empirical evidence. 

Therefore, this foresight approach to scenario studies must seek a balance between consensus 
and expertise, between asking the ‘usual suspects’ and bringing in new people with fresh ideas 
and between creativity and evidence when thinking about possible developments. Outsiders 
bring in new ideas, but these may not resonate with insiders in the organization; newbies pro- 
vide a fresh look at a topic but often lack the insight into the basics concepts and structures of 
a problem; thinking outside the box is important, but many scenario studies eventually have to 
resonate with people working ‘inside the box’ as well. 

Theoretically, these problems also apply to the forecasting approach, but since forecasting 
relies more on scientific evidence and is done from a scientific and positivist tradition, it is hardly 
an issue in practice. In the case of foresight, the balance between creativity and evidence is less 
clear; the point of doing a foresight study is to look beyond what is currently known towards 
unexpected, not yet adequately researched and not completely understood trends. That inher- 
ently involves creativity and uncertainty but also requires a basis for credibility and quality. 

Scenarios can be developed at many different levels of analysis (Dammers 2000). They can be 
limited to a single sector (e.g., transport) but also cover multiple sectors. In fact, futures thinking 
can be a wedge to break silos and cut across fixed concepts of domains and sectors. The level of 
abstraction for a scenario is also important: does the study focus on the micro (unemployment), 
meso (social welfare) or macro (economic structure and societal norms and values) level? Is it 
about the organization, the sector or the broader environment? Another choice to make is the 
level of exploration and speculation for the scenarios. Does the scenario primarily follow cur- 
rent trends, established relations and existing policies? Or should it look beyond current patterns 
and seek out discontinuities? A further choice is between ‘prospective’ or ‘projective’ scenarios. 
Projective scenarios are based on extrapolations and continuations of current trends to look into 
the future. Prospective scenarios start from the future, look at possible futures and then reason 
back to relate the scenarios to the present (‘backcasting’). Both options produce narratives of 
how and why the present developed into possible futures, but the direction for producing those 
narratives is different. A projective scenario ‘argues forward in time’ while a prospective scenario 
reconstructs the path towards a scenario ‘back from the future’. 

Scenarios can be subdivided into two types; policy scenarios and environmental scenarios. 
A policy scenario describes possible variants or ‘worlds’ for a specific policy, whereas an envi- 
ronmental scenario describes different possible contexts in which a variety of policies can be 
‘tested’. Environmental scenarios can be made once and then used for different policies; they are 
nonspecific to a particular type of policy. Different economic scenarios can be used to test all 
sorts of policies, from educational policy to employment policy and traffic congestion. This use 
of environmental scenarios is referred to as wind tunnelling; policy proposals are tested for their 
ability to perform in different possible future environments. 

A final choice for scenario builders is the amount of ‘policy intelligence’ to build into the 
scenarios. Traditionally, scenarios are considered policy neutral; this means that in building the 
scenarios, there is no room for policymakers to change course when they see matters become 
worse. They do not correct along the way or take measures to compensate for negative effects 
of the scenario. Therefore, policy-neutral scenarios typically produce ‘extremes’, and a frequent 
critique is that such extremes will never occur in practice because policymakers correct along 
the way. Scenario builders often argue that the extremes help policymakers see the route they are 
on while critics counter that the extremes of policy-neutral scenarios result in extreme cases that 


250 


Analysis, Forecasting, Backcasting 


‘real’ policymakers do not take seriously because they consider them too unrealistic. However, 
it is perfectly possible to design mildly policy-rich scenarios and take the learning capacity of 
policymakers at least somewhat into account — for instance, by ‘allowing’ policymakers in the 
narrative of the scenario to correct the most negative aspects of the scenario and ‘make the most 
of it’ within the bounds of the driving forces. 

There are many other methods for futures studies apart from scenario studies (Bell 2003; 
Slaughter 2005; Glenn and Gordon 2006). These methods can be applied in the approaches of 
both forecasting and foresight. We will briefly discuss several of them. 

Trend extrapolation: researchers set out values on a timeline and extrapolate these historic 
trends into the future. The extrapolation can be linear or more dynamic, and often, a variety 
of deviations from the trends are also projected. This creates a ‘channel’ in which the trend will 
probably move but also leaves room for variation. 

Modeling: researchers construct a model of a real-world social system in which the different 
variables that are considered typical for the system are related to each other. Variables respond to 
each other, and the models can be highly sophisticated; some even argue that is nearly possible 
to build models that actually represent the reality of complex systems, but this remains contested. 
Models usually work in ‘rounds’ that represent fixed time periods; by running multiple rounds, 
the future of a system can be simulated. Different types of interactions of actors and factors can 
be simulated to see the variety of options a system can produce. This can be done deliberately 
to explore certain directions, but also in a Monte Carlo—type random selection. The model 
provides the researchers with a range of possible outcomes of the actual social system in a greater 
level of detail. 

Scanning: The researcher analyses a large pile of possible sources to identify possible pat- 
terns. Sources may vary greatly, from mainstream news media such as newspapers, magazines or 
television programs to obscure blogs and Twitter messages. Researchers can also look for more 
scientific sources, including expert websites, publications and academic research programs that 
reflect the development in expert fields. 

Emerging issues: This type of analysis somewhat resembles ‘scanning’ but deliberately sets 
out to look beyond the current lines of reporting. The idea is to look beyond the current dis- 
course to notice possible disturbances or signs of new developments early on. In hindsight, most 
unexpected ‘path-breaking’ developments had already been developing for years outside the line 
of sight of ‘normal’ research. Usually, the identification of emerging issues requires looking in 
different places, talking to different people and looking at other types of sources. For example, 
some researchers look at art house movies, visit new places, read blogs or invite groups they 
usually would not talk to. 

Signposting: This method identifies triggers, signs or thresholds of the emergence of a pos- 
sible future that can act as a warning that a particular scenario is becoming reality (Walker et al. 
2012, 2013; Splint and Van Wijck 2012). Signposting includes an overview of several possible 
futures that can be translated into pathways for which signs can be distinguished. The signs can 
help warn policymakers early so they can respond in a timely fashion. At the same time, naming 
the signs also generates awareness of the possibility — or even inevitability — of surprise. 


Trends and Developments in the Fields of Futures Studies 


One development in the world of forecasting is that the dramatically expanding capacity of 
computer power is enabling wider and ‘deeper’ models. Forecasters argue that this allows them 
to do two things: they can map more elaborate paths and become more accurate in the devel- 
opments they forecast. Widening the forecast means that more variables and relations can be 
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entered into the model to provide more detail in the model or to model wider ground. On the 
issue of accuracy, forecasters point at the now-emerging ability to map more complex relations 
and to consider many or most — or some argue ‘all’ — possible options of a causal relation (Bloe- 
men et al. 2014; Bloemen et al. 2019; Haasnoot et al. 2013, 2012; Groves and Lempert 2007). 

Furthermore, if computer capacity grows even further, and transaction costs of calculation 
become almost nonexistent, forecasters argue that they can consider all possible dynamics in a 
complex system. This would mean that they can model all possible futures of a complex system 
and forecast all the possible consequences — through all the possible causal interactions — of an 
intervention. Extrapolation remains the principle of forecasting, but because the extrapolation 
can be done in with a limitless number of options, the extrapolation can be ‘whole’. The hypo- 
thetical outcome of this development is a model that is not predictive in the sense that it “knows’ 
the future but that is able to map all the possibilities of a system. 

It is important to note that this relates to fundamental debate about complexity; can a com- 
plex system model be modeled in full, or will emergence and reflexivity inherently produce 
unforeseen turns and developments in the system that render each model at some point limited 
and outdated? 

There are also developments in the world of foresight. One of the problems of the typical 
two-axes scenario method is that although it opens a debate about the future, it inherently nar- 
rows it also; the method reduces the future to two so-called ‘core uncertainties’ that produce 
four very distinct directions for the future or future worlds (Van Asselt et al. 2007; Van der Duin 
and Van der Steen 2012). Even though this opens up the thinking about possible futures, it also 
lays a heavy burden on the choice for the core uncertainties (Van der Steen 2016). For most 
wicked policy issues, there are not two but many more ‘core uncertainties’. A more fundamental 
version of that same critique is that it is inherently uncertain what the core uncertainties should 
be. Some will argue that this is beside the point of a foresight study; accuracy is not the purpose 
of such a process; it is the active engagement of a group with possible futures that matters. At the 
same time, it is evident that the conversation is guided by the choice of scenarios; that is why, 
in the field of foresight studies, attempts are made to find new means to systematically cover a 
wider range of uncertainties and ignite a broader and richer conversation about possible futures 
(Van der Duin and Van der Steen 2012). 

Interesting also are attempts to link foresight to new technologies such as virtual reality and 
serious gaming but also to other modes to make scenarios experiential (e.g., Selin et al. 2015; 
Rikens-Klomp 2012). Such methods allow for a deeper and more interactive ‘experience’ of 
possible futures that moves the process from an intellectual conversation about the future towards 
a more action-oriented one with possible futures (Van der Steen 2016; Rajkens-Klomp 2012; 
Wiebe et al. 2018). Examples of this are simulations in which adults can ‘experience’ a day in 
their current home but in the conditions of a seventy-year-old with minor disabilities; this is part 
of program to incentive adults to be more aware of financial preparations for old age and, more 
specifically, the benefits of building or redesigning homes in such a way that they can accom- 
modate future modifications easily. This is an example of an issue that people understand on a 
cognitive level but that only becomes ‘real’ and consequential when people experience it for 
themselves and in their own home. It is not enough for them to see images of possible futures 
or to study a spreadsheet or text that tells them about it; they need to ‘live’ the future(s) in order 
to feel the urgency to act and to weigh options for doing so. 

Also interesting are the attempts to apply the repertoire of forecasting to massive amounts 
of qualitative data: e.g., by using software to analyze patterns in narratives or in images about 
the future, in media reports, social media posts or accounts by large populations of ‘ordinary’ 
people who use self-reporting apps (e.g., the Cynefin Sensemaker App). That allows for the 
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development of ‘big meaning’, the use of qualitative data to find apparent patterns in how 
people see possible futures and to find weak signals or early warnings of emergent developments 
(Van der Steen et al. 2013). That may allow researchers to see patterns in highly complex pat- 
terns through a combination of forecasting methods and qualitative data. 

An important challenge for the field of foresight is how to navigate the balance between 
creativity and facts in a world where post-truth, alternative facts, and fact-free thinking have 
become more common (Nehme et al. 2012; Rajkens-Klomp 2012; Hoppe 2010). Foresight 
inherently involves a combination of fact and fiction, in the sense that participants and research- 
ers develop possible futures in a creative process that looks beyond mere facts (Nelson et al. 2008; 
Cairns et al. 2006). Discontinuity requires imagination, and even though that might be done in 
a systematic manner, it remains a problematic basis for policy study and political debate (Van der 
Steen and Van Twist 2013; Van der Steen et al. 2015; Van der Steen 2008; Groombridge 2006). 
What exactly discerns a foresights study from a politically motivated ‘alternative truth’? Foresight 
studies cannot do without creativity and imagination but must at the same find an anchor to 
show for validity and methodological rigor. 

A final noteworthy development in the field of forecasting is the attempt to tighten the link 
between possible pathways of the system and required policy interventions (Haasnoot et al. 
2013, 2012; Walker et al. 2013; Bloemen et al. 2010; Bloemen 2018; Biesbroek et al. 2010; 
McCray et al. 2010). 

The adaptive pathways approach to climate adaptation policy in the Netherlands is an exam- 
ple of this. Extensive forecast models have projected a set of possible paths of the climate system 
(in this case, in the Netherlands) and have identified signals and signposts that show on which 
of the paths reality is developing (Bloemen 2018; Bloemen et al. 2019). If signals show that the 
system is moving onto a different pathway, an ‘automated’ policy response is triggered that sets in 
motion the predefined required policies for that particular path. The idea of this concept is that 
not only the development of the system, but also the package of policy interventions is prede- 
signed and optimized by using models to ‘test’ for its consequences (Walker et al. 2010). 

This is a further extension of the use of forecasting models, beyond mere ‘knowledge for 
policy’ into the model as policy; policy, as an automated response, is designed into the model 
(Haasnoot et al. 2012). This brings policymaking up to par with other automated ‘decisions’, 
such as automated trading on the stock market or automated safety locks in critical systems. At 
the same time, this is also a problematic and highly political development since the decision- 
making is transferred into the forecast model. 


Conclusion: Using Systemic Studies of the Future in Policy 
Formulation and Decision-Making 


Thinking about the future is inherently done in the plural. There is not one future; there are 
many possible and plausible futures. The purpose of good anticipatory work is to generate a 
range of possible futures that are beyond the continuation of the patterns of the past and pres- 
ent. These patterns are important and, to some extent, will shape the future, but futures are 
not limited to these patterns; the future is “open but not empty’ (WRR 2010). The future is an 
infinite number of possibilities which are not equally likely, plausible or preferable. The keys to 
conducting good anticipatory work are to take discontinuities and the width of possibilities into 
account, to be able to prioritize and filter out the most important options and to relate these 
options meaningfully to the current discourse in policy formulation. 

Systematically studying the future can provide important insights for policy formulation and 
decision-making. This can be done directly, when futures studies provide ‘knowledge for policy’, 
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but also indirectly, when the participation of policy makers in anticipatory work opens up their 
time horizon and makes them more inclined to take future dynamics into account. Studying 
the future work can help policymakers come up with better solutions for policy problems; more 
importantly, it helps them identify the problems, solutions and working conditions of the future. 

In this chapter, we have presented a variety of methods used to systematically study unknown 
futures as part of the policy process. These methods are well constituted in a professional body 
of knowledge and practice, and some of them are described in quite strict terms. Taking these 
methods seriously is important for doing future studies. 

However, as Bell (2003) argues, tools and methods are only as good as the person or team 
that is applying them. In the end, good future-oriented policy work inherently depends on the 
person or team doing it. Imagination, analytical capacity, leadership and listening capacity are 
all part of the varied skillset that is required to conduct good anticipatory work. Good anticipa- 
tory policy work needs professionals who understand both the context of futures studies and the 
dynamics of the policy process. 
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FORMAL PROBLEM 
STRUCTURING TOOLS 
AND MULTI-CRITERIA 

DECISION ANALYSIS 


Mika Marttunen, Judit Lienert and Valerie Belton 


Structuring problems for multi-criteria decision analysis (MCDA) has attracted increasing atten- 
tion over the past 20 years from both a conceptual and a practical perspective. In this chapter, we 
describe a set of widely used formal problem structuring tools and how they can be applied to 
frame and analyze complex decision situations. The following methods are considered: cognitive 
and causal maps, DPSIR diagrams, scenario planning, stakeholder analysis and SWOT. We also 
describe how these methods can be combined with MCDA and discuss experiences and chal- 
lenges related to their joint use. 


Introduction 


Structuring problems for multi-criteria decision analysis (MCDA) has attracted increasing atten- 
tion over the past 20 years from both a conceptual and a practical perspective as a tool for aiding 
decision-making. This is reflected in a significant growth in the number of published applications 
which use a formal approach to problem structuring in combination with an analytic method for 
multi-criteria analysis. The combined use of problem structuring methods (PSMs) and MCDA 
was investigated in an extensive review by Marttunen et al. (2017) and is summarized here. 

In total, 333 articles published during 2000-2015 were identified and analyzed in the article. 
From these, 68 articles covering all the potential PPM-MCDA combinations were selected and 
studied in detail. The study aimed to answer the following questions: 


How common is the joint use of the methods? 

How are they combined, and how is the information produced by PSMs used in MCDA? 
What are the key benefits of these combinations, and what problems have been reported? 
How can MCDA practices be enhanced by using PSMs? 


The literature review covered in the article identified eight PSMs (cognitive and causal maps, 
DPSIR, scenario planning, SSM, stakeholder analysis, strategic choice approach, SODA and 
SWOT (see explanations in Table 21.1) and seven MCDA methods (AHP, ANP, ELECTRE, 
MAUT, MAVT, PROMETHEE and TOPSIS; see explanations in Marttunen et al., 2017) used 
or promoted as effective means to improve decision-making. 
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Table 21.1 List of Problem Structuring Methods and the References 


Method/Methodology 


Description 


Reference 


Cognitive maps 
(CMs) and 
group maps 
(GMs) 


DPSIR framework, 
PSR framework 


Scenario planning 


(SP) 


Soft systems 
methodology 
(SSM) 


Stakeholder analysis 


Strategic 
assumptions 
surfacing and 
testing (SAST) 


Strategic options 
development and 
analysis (SODA) 


SWOT analysis 


A CM is a graphical representation which captures how 
an individual perceives a particular issue in terms 
of key aspects of the system and perceived causal 
relationships between these, with the aim of improving 
understanding and informing decision-making. A 
group map is the integration of a number of individual 
CMs or a map created in a facilitated process involving 
a number of people. 

A causal framework for describing the interactions 
between society and the environment. DPSIR stands 
for driving forces, pressures, state, impact, responses. 
An extension of the PSR framework used by the 
OECD. 

Scenario planning, also called scenario thinking or 
scenario analysis, is a strategic planning method to 
identify and analyze plausible but not necessarily 
probable or desirable futures and to use these to help 
identify appropriately flexible long-term strategies. 

Action-oriented process of inquiry into a problematic 
situation using different methods to structure the 
discussion and enhance learning. Commonly used 
methods are Rich pictures, root definitions; customers, 
actors, transformation, worldview, environment 
(CATWOE); PQR (What, How, Why) and 3 Es 
(efficacy, efficiency, effectiveness). 

Process of identifying the individuals or groups that are 
likely to affect or be affected by a proposed action. 
Results in specific participation strategies for each 
group. 

Aims to surface, map and evaluate underlying 
assumptions that managers (stakeholders) bring with 
them (often subconsciously) in relation to an issue 
of strategic concern and to examine the relationship 
between these assumptions and potential policies with 
a view to formulating more robust and potentially 
novel or previously unforeseen policies. 

Supports a group to construct a shared graphical 
representation of a problematic situation as a causal 
map (a group map, or GM) and to use this to explore 
potential strategies with respect to a complex system 
of goals. A GM can be created by merging the CMs of 
a number of individuals or by using Oval mapping in a 
workshop environment. 

Tool for identifying criteria of the internal (strengths, 
weaknesses) and external (opportunities, threats) 
operational environment and for determining 
strategies. 
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Eden (1992) 


OECD (1993) 
EEA (1995) 


Schoemaker 
(1995) 


Checkland and 
Scholes (1990) 


Grimble and 
Wellard (1997) 


Mitroff and 
Emshoff (1979) 


Ackermann and 
Eden (2010) 


Sarsby (2016) 


Formal Problem Structuring Tools 


This chapter summarizes some of the main results of the review regarding five of the seven 
PSMs studied in the original article: namely, cognitive mapping, DPSIR, scenario planning, 
stakeholder analysis and SWOT (Table 21.1). Some of these PSMs are well known in OR/ 
management science (e.g., cognitive mapping), whereas others have not received much atten- 
tion (e.g., DPSIR). It should be noted that different PSMs use different terms for issues which 
are identified and structured; we use the term ‘criterion’ as a synonym for factor, variable and 
objective. To combine MCDA with PSMs in practice, a good understanding of the basics of 
MCDA is needed. We refer interested readers unfamiliar with MCDA to textbooks such as Bel- 
ton and Stewart, 2002 or Eisenfiihr et al., 2010. 


Multi-Criteria Decision Analysis Methods 
and Need for Problem Structuring 


The first stage in any decision process is the understanding of the broader context and charac- 
teristics of the problem (Figure 21.1). This includes the identification of stakeholders and their 
interests, concerns and hopes. During this diverging or opening-up phase, a comprehensive 
representation of the problem is developed which may be presented, for example, in the form 
of a cognitive or causal map, a means-end network or a value tree. MCDA is appropriate for 
well-structured problems. It does not, per se, include any specific tools for the problem struc- 
turing phase, although nowadays, it is increasingly applied in concert with problem structuring 
methods (Marttunen et al., 2019). 

In earlier MCDA literature, the problem structuring phase generally received less attention than 
the MCDA evaluation. This began to change in the late 1980s (Belton and Stewart, 2010; Franco 
and Montibeller, 2010). A particularly notable development which had significant impact on the 
field was value-focused thinking (VFT) (Keeney, 1988, 1996). VFT is a holistic approach which 
incorporates a systematic procedure to support the identification and structuring of the decision- 
makers’ values and objectives as well as the creative generation and evaluation of alternatives. 
Another holistic methodology is structured decision-making (SDM) (Gregory et al., 2012). SDM 


Divergence Convergence 


Understanding the problem Evaluating alternatives 
Generating alternatives Agreeing a way forward 
Finding out about people’s Assigning responsibilities 
concerns Allocating budgets 


Creating framework for evaluation 


Building a value tre 


Problem 
structuring on 


MCDA 


TPP C PSEC < 


Figure 21.1 Diverging and Converging Phases of Decision-Making and Stages in Building a Value Tree 
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embeds several tools and practices for supporting a collaborative application of MCDA, combining 
methods from decision analysis and applied ecology with insights from behavioral research. 


Problems Structuring Methods and MCDA 


Stakeholder Analysis and MCDA 


MCDA is increasingly seen as a powerful approach to support collaborative processes (e.g., 
Lennox et al., 2011; Marttunen et al., 2015) and the review of Marttunen et al. (2017) indi- 
cates that stakeholder involvement in MCDA is now common, particularly in environmental 
planning. Several MCDA approaches emphasize the importance of stakeholder participation: 
namely, the decision analysis interview approach (Marttunen and Hämäläinen, 1995, 2008), 
stakeholder multi-criteria decision aid (Banville et al., 1998) social multi-criteria evaluation 
(SMCE) (Munda, 2004), deliberative multi-criteria evaluation (Proctor and Drechsler, 2006) 
and multi-actor multi-criteria analysis (MAMCA) (Macharis et al., 2009). 

In social multi-criteria evaluation (SMCE), there is a strong focus on understanding social 
actors’ needs and their interrelationships in the problem structuring phase (Munda, 2004), and a 
careful stakeholder analysis is a vital part of SMCE (e.g., Gamboa, 2006; Borzoni et al., 2014). 
Lienert et al. (2013) combined a thorough stakeholder analysis (Grimble and Wellard, 1997) 
with a social network analysis (Kenis and Schneider, 1991), using the results in a participatory 
MCDA for water infrastructure planning (Lienert et al., 2015). 

Stakeholder identification and analysis is highly relevant in MCDA because it explicitly rec- 
ognizes multiple actors and their objectives (Table 21.2). MCDA methods do not explicitly 
incorporate any techniques for this task. It is important to be aware that engaging stakehold- 
ers always raises an ethical question related to the treatment of the different stakeholders in an 
MCDA process (Banville et al., 1998). 


Cognitive Maps, Group Maps and MCDA 


Cognitive maps (CMs) and group maps (GMs) have been applied primarily in psychology and 
the behavioral sciences, management, politics and economics (Kpoumi€é et al., 2012). They have 
been increasingly used to structure and help explain complex social and environmental problems 
and associated decision-making (Tikkanen et al., 2006). CMs are generally records of verbal 
accounts of issues given in interviews. They are networks of ideas linked by arrows; the network 
is coded from what a person says (Eden and Ackermann, 2004). 


Table 21.2 The Pros and Cons of Stakeholder Analysis and Its Combination with MCDA 


Pros and Cons How Stakeholder Analysis Challenges and Issues to Be 
Benefits MCDA Aware of When Combining 
+ Easy to use. Helps identify key stakeholders Decision about who should 
+ Systematic if done properly. to be engaged in MCDA. be engaged in MCDA can 
— Subject to criticism for lack Encourages consideration of still be difficult. 
of academic rigor. a range of perspectives. 


— Difficult if little initial 
knowledge of stakeholders. 


(+ positive aspect, — negative aspect) 
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Table 21.3 The Pros and Cons of Cognitive Maps (CMs)/Group Maps (GMs) and Their Combination 


with MCDA 
Pros and Cons How CMs/GMs Benefit MCDA Challenges and Issues to Be 
Aware of When Combining 
+ Helps surface and structure an Improved understanding Defining a value tree 
individual’s or a group’s collective of problem and different directly from the 
ideas using people’s natural language. perspectives can help define CM/GM can be 
+ Helps explain different perspectives. the goal of MCDA, build challenging and/or 
— Skillful facilitator(s) needed in a value tree and develop laborious. 
complex cases. alternatives. A detailed map 
— Specialist software needed to analyze can serve as ‘organizational 
large maps. memory’ and support use 
— Merging individual cognitive maps to of a simpler MCDA (e.g., 
generate a group map is challenging. smaller value tree). 


(+ positive aspect, — negative aspect) 


CMs/GMs and MCDA methods have typically been deployed alone (Franco and Lord, 2011). 
The joint use cases identified in Marttunen et al. (2017) relate to business, environmental and 
social issues, covering both the public and private sector. MCDA benefits from CMs/GMs in 
several ways (Table 21.3): CMs/GMs stimulate participants to structure their ideas using natural 
language (e.g., Montibeller et al., 2008); they facilitate communication and problem structuring 
(Wolfslehner and Vacik, 2011); they enable people to voice their views and even to persuade 
others (Franco and Lord, 2011). CMs/GMs can also reduce framing biases (Hodgkinson et al., 
1999, and comments by Wright and Goodwin, 2002; Hodgkinson et al., 2002). 


DPSIR Framework and MCDA 


The pressure-state-response (PSR) framework was firstly proposed by the OECD for analyzing 
environmental issues in 1970. Its extension, the driving forces-pressure-state-impact-response 
(DPSIR) framework was developed in the 1990s to help better structure and organize environ- 
mental criteria and environment-society interconnections to assist policymakers in an under- 
standable way (OECD, 1993; Smeets and Weterings, 1999). 

DPSIR has been increasingly used for environmental issues and research projects (Meyar- 
Naimi and Vaez-Zadeh, 2012; Tscherning et al., 2012). Because DPSIR simplifies complex 
environmental relationships, it has been subject to several criticisms (Tscherning et al., 2012; Gari 
et al., 2015). Subsequently, it has been modified in several ways: for instance, by adding a welfare 
element (Cooper, 2013; Kelble et al., 2013; O’Higgins et al., 2014). Alternatively, DPSIR has 
been integrated with other methods, such as system dynamic modeling (Lee and Lin, 2014). 

Most identified DPSIR-MCDA cases in Marttunen et al. (2017) relate to water or wetland 
management; others concern forest management, urban or land use planning and environmen- 
tal status assessments. DPSIR can provide a powerful framework to aid initial understanding, 
structuring and communication of a complex environmental issue (Table 21.4). DPSIR helps 
visualize cause-effects and improves communication (Wolfslehner and Vacik, 2011; Bottero and 
Ferretti, 2010; Johnston et al., 2013). 

However, it is limited with regard to a rigorous scientific analysis, and the integration with 
MCDA can be challenging because the DPSIR framework as such cannot be used. If the 
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Table 21.4 The Pros and Cons of the DPSIR Framework and Its Combination with MCDA 


Pros and Cons How DPSIR Benefits MCDA Challenges and Issues to Be 


Aware of When Combining 


+ Helps identify key Supports criteria identification. Difficult to derive criteria from 


relationships between 
environment and society. 

— Deciding to which category 
a criterion belongs (e.g., 
state or impact) can be 
difficult. 

— Unidirectional cause- 
and-effect relations (no 


In particular, impact criteria 
are good candidates for a 
value tree. Thinking about 
driving forces, pressures, 
states and impacts supports 
development of alternatives. 
DPSIR provides a useful 
framework to calculate an 


DPSIR criteria. Need to be 
selective in using DPSIR 
criteria in a value tree. 
Uncritical use of criteria can 
result in double counting of 
criteria, which can provide 
wrong or misleading MCDA 
results when aggregating 


over criteria. 


feedback loops). index. 


(+ positive aspect, — negative aspect) 


whole framework is used in sustainability assessments, we suggest that the results of different 
DPSIR categories be presented separately and not aggregated, unless categories are thoroughly 
rechecked by MCDA specialists to avoid double counting. 


SWOT and MCDA 


SWOT (strengths, weaknesses, opportunities and threats) was originally developed to 
inform the generation of organizational strategies. SWOT and MCDA have been com- 
bined in many application areas, including agriculture and forestry (Shrestha et al., 2004), 
bioenergy (Catron et al., 2013), information technology (Kahraman et al., 2007), tourism 
(Akbulak and Cengiz, 2014), traffic (Bottero, 2015) and water resources management (Srd- 
jevic et al., 2012). 

Marttunen et al. (2017) identified four different ways of combining SWOT and MCDA: (1) 
SWOT analysis is first carried out, relevant internal and external criteria are identified and their 
relative importances are determined using pairwise comparisons (e.g., Duchelle et al., 2012; 
Catron et al., 2013); (2) SWOT criteria are prioritized and used in the development of manage- 
ment strategies (Oztiirk, 2015); (3) SWOT framework is used to evaluate existing alternatives 
(Scolozzi et al., 2014; Ekmekcioglu et al., 2011); (4) alternatives derived from SWOT are evalu- 
ated using MCDA (e.g., Terrados et al., 2009; Sevkli et al., 2012). 

The combination of SWOT with MCDA is mutually beneficial and provides an effective 
framework in strategic decision-making (Kurttila et al., 2000). SWOT can benefit MCDA in 
several ways. Firstly, SWOT can bring added value to stakeholder involvement, supporting the 
development of a common language and providing a simple method to improve communication 
and learning (Kurttila et al., 2000; Kajanus et al., 2004; Nikodinoska et al., 2015). Secondly, 
SWOT helps better explain the decision situation and its underlying structure. Thirdly, it ensures 
that all relevant aspects are considered through the analysis of all SWOT criteria from both an 
internal and an external viewpoint. Fourthly, SWOT supports developing new strategies or 
alternatives using a TOWS matrix (Weihrich, 1982; Dyson, 2004), which confronts the ele- 
ments of internal quadrants (strengths, weaknesses) with those of external quadrants (opportuni- 
ties, threats) (e.g., Terrados et al., 2009; Sevkli et al., 2012). A SWOT-MCDA combination is 
also useful for visualization; in the four-quadrant SWOT diagram, the x-axis refers to internal 
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Table 21.5 The Pros and Cons of SWOT Analysis and Its Combination with MCDA 


Pros and Cons How SWOT Benefits Challenges and Issues to Be 
MCDA Aware of When Combining 

+ Easy to use, widely known. Supports criteria Can be difficult to transform 

+ Thinking about internal and external identification and SWOT criteria to be 
criteria improves overall understanding development of appropriate for a value tree. 
of decision situation. alternatives Need to be selective in using 

— Provides no means to determine the (using TOWS matrix). SWOT criteria in a value 
relative importance of criteria, which is tree. May not generate all 
needed for objectives used in MCDA. relevant criteria. 


(+ positive aspect, — negative aspect) 


criteria (strengths, weaknesses) and the y-axis to external criteria (opportunities, threats), which 
can inform the development of the MCDA model (e.g., Kurttila et al., 2000). 

The SWOT-MCDA approach also has shortcomings (Table 21.5). Deciding which SWOT 
group a criterion belongs to can be challenging. Ghazinoory et al. (2007) found that the inter- 
nal and external criteria cannot always be classified as purely positive or negative because they 
contain both types of effects. These shortcomings of SWOT may have implications for the fol- 
lowing MCDA. Another challenge for MCDA is how to operate with a high number of criteria 
which may be identified in SWOT if criteria are used as objectives. To avoid double counting 
of objectives or incorporating means objectives in MCDA (see textbooks, e.g., Eisenfiihr et al., 
2010), SWOT criteria should be further processed (e.g., using PESTLE categories (political, 
economic, social, technological, legal, environmental) (Srdjevic et al., 2012) or a value-focused 
thinking approach (Kajanus et al., 2004). 


Scenario Planning and MCDA 


The integrated use of scenario planning and MCDA (the combined use of scenario analysis 
and multi-objective assessment) has been addressed in several articles (e.g., Durbach and Stew- 
art, 2003; Montibeller et al., 2006; Ram et al., 2011; Karvetski et al., 2011a; Karvetski and 
Lambert, 2012; Stewart et al., 2013; Scholten et al., 2015; Ram 2020). The two approaches 
are complementary (Wright and Goodwin, 1999; Montibeller et al., 2006), and there are 
many mutual benefits when applied jointly. For instance, MCDA does not adequately deal 
with the many uncertainties that arise, especially in long-term strategic decision-making con- 
texts, which, in turn, is the strength of scenario planning (Stewart et al., 2013). Stewart (1997, 
2005) presents several technical issues and a thoughtful discussion concerning this integration, 
and Stewart et al. (2013) give a good overview of the mutual benefits. 

Most studied cases combined qualitative participatory methods with quantitative models. 
Mostly a ‘ful? MCDA was also realized, including assigning weights to the criteria and calculat- 
ing overall priority values for the alternatives. However, there are large differences in the realiza- 
tion of these phases and in the choice of MCDA methods. Some examples include stakeholder 
analysis (Lienert et al., 2015), SWOT (Leskinen et al., 2006) and value-focused thinking (Mon- 
tibeller et al., 2006) to support the structuring phase, and MAVT (Ram and Montibeller, 2013; 
Scholten et al., 2015; Montibeller et al., 2006), AHP (Leskinen et al., 2006) and PROMETHEE 
(Kowalski et al., 2009) to evaluate the alternatives in different scenarios. 
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Table 21.6 The Pros and Cons of Scenario Planning and Its Combination with MCDA 


Pros and Cons How Scenario Planning Benefits MCDA Challenges and Issues to Be 
Aware of When Combining 


+ Encourages thinking about Can broaden scope of MCDA to Interpretation and 
different possible futures. analyze problems with long time elicitation of criteria 

+ Challenges people’s horizons and encourage creativity importance weights 
conventional thinking. in developing new alternatives. can be challenging 

— Design of scenarios can be Scenarios can be used to explore (depends on approach). 
demanding and laborious. the robustness of alternatives. 


— No ‘inbuilt’ tools for 
comparing alternatives. 

— Skillful facilitator needed 
in complex cases. 


(+ positive aspect, — negative aspect) 


The integration of MCDA and scenario analysis is promising but methodologically chal- 
lenging (e.g., Kowalski et al., 2009; Ram and Montibeller, 2013). The combination adds an 
additional dimension to the already-extensive preparation required for scenario planning (e.g., 
Bizikova and Krcmar, 2015) and to potentially complex MCDA analyses (e.g., Lienert et al., 
2015) (See Table 21.6). Possible comparisons of the outcomes of alternatives in each scenario 
may be time consuming and cognitively demanding (e.g., Montibeller et al., 2006; Ram et al., 
2011). One difficulty in assessing the performance of strategies in scenario planning is that often 
they consist of sub-options which have to be considered simultaneously (e.g., Montibeller et al., 
2006). Inclusion of the different perspectives of multiple decision-makers in group negotiation 
can add to the challenge (Ram et al., 2011). For instance, Scholten et al. (2015) carried out 40 
independent MCDA calculations, one for each of four scenarios and each of ten stakeholders. 
To improve the efficiency of MCDA and to reduce the cognitive load of participants, Kar- 
vetski et al. (2011a, 2011b) developed an approach which simplifies the elicitation of preference 
weights, as the entire value function is not totally reconstructed for each scenario. 


Discussion 


Different Ways to Combine PSMs With MCDA 


These methodologies and methods can be combined in several ways, as discussed in the general 
OR/MS literature (Mingers and Brocklesby, 1997; Kotiadis and Mingers, 2006; Mingers, 2007; 
Belton and Stewart, 2010). The main distinctions are whether the methodologies come from 
the same or different paradigms, the number of methodologies/methods used, whether whole 
methodologies are used or parts and how they are integrated. 

Typically, the PSMs and MCDA are applied sequentially or in an integrated manner. When 
used sequentially, one or more PSMs inform the subsequent MCDA. This is most commonly 
used and, we found many examples. Examples applying a single PSM with MCDA include 
stakeholder analysis (Nordstrom et al., 2010), SWOT (Kurttila et al., 2000), DPSIR (Chung and 
Lee, 2009) and cognitive mapping (Bana e Costa et al., 2006). In 19 articles multiple PSMs were 
used (e.g., Petkov et al., 2007; Bana e Costa et al., 2014; Lienert et al., 2015). 


264 


Formal Problem Structuring Tools 


The combination of scenario analysis and MCDA moves from a more independent consid- 
eration of the two perspectives to an integrated analysis. Initially, the options and the evaluation 
criteria are identified using MCDA; the scenarios which anticipate potential futures are con- 
structed using scenario analysis. In the subsequent integrated analysis, the options are evaluated 
in the context of each scenario, and overall performances across scenarios are compared. Several 
SWOT-MCDA cases also used the two methods in an integrated way (see details in Marttunen 
et al. 2017). 


The Contribution of PSMs to Problem Structuring for MCDA 


Different methods have different purposes, and therefore several methods may be needed to 
cover all phases of problem structuring (Table 21.7). 

Defining the frame: All methods support this phase to some extent. CMs and GMs are 
especially useful in complex decision situations as they are specifically designed to outline the 
decision situation and to support the formation of a shared understanding. 

Identifying stakeholders: Stakeholder analysis provides a systematic way to identify key stake- 
holders and to define their roles and perspectives. 

Identifying criteria: Cognitive and group maps have been used to develop a comprehensive set 
of criteria. They can also help identify fundamental objectives (which answer the question ‘Why 
is that important?’; see Keeney, 1996) and distinguish them from means objectives (which answer 
a ‘How can this be achieved?’ question). Likewise, SWOT and DPSIR can help identify relevant 
criteria in a studied system. However, these criteria have to be transformed to fundamental objec- 
tives, and means objectives need to be excluded before they can be used in a value tree. 

Developing alternatives: The DPSIR framework can stimulate thinking about alterna- 
tives because responses are considered in four different levels: driving forces, pressures, states 
and impacts. SWOT criteria can be used in the systematic generation of alternatives through 
the use of SWOT quadrants. Scenario planning can encourage creativity in developing new 
alternatives. 


Table 21.7 Level of Support Provided by PSMs to Different Aspects of Problem Structuring for MCDA 


Stakeholder Analysis | Cognitive Map/ Scenario Planning 


Defining the frame 


Identifying stakeholders 


Identifying criteria 


Developing alternatives 


Identifying uncertainties 


No support 


Some support 


Moderate support 


Strong support 


(Estimates are tentative and reflect the achieved synergies if PSMs are used as typically described.) 
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Identifying uncertainties in the external environment: External SWOT criteria (opportuni- 
ties and threats) can provide insights to scenario development. Each future scenario can have its 
own SWOT analysis. Scenario planning is a powerful method to explore external uncertainties. 


Conclusions 


Different PSM tools have different purposes, and therefore several methods are needed to address 
all phases of problem structuring. PSMs and MCDA are complementary methods, and when 
applied together, there are many synergies and mutual benefits. Combining PSMs and MCDA 
tools produces a richer view of the decision situation and provides a methodology which can 
better handle the various phases of decision-making. Identifying PSM-MCDA combinations 
which are most effective in specific decision situations is an important research topic. 

There are also some limitations and challenges in combining PSMs and MCDA, most impor- 
tantly relating to building a value tree and assigning importance weights to the criteria. The 
potential benefits of combining PSMs and MCDA methods are not yet fully recognized among 
MCDA practitioners and researchers, and we encourage our colleagues to further explore this 
in their work. 
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SIMULATION MODELING 
AS A POLICY TOOL 


Bernardo Alves Furtado 


This chapter describes, justifies, presents the pros and cons of and illustrates the use of simula- 
tion modeling as a handy, cost-effective and agile tool for policymakers. Simulation modeling 
is flexible enough to accommodate different levels of detail, precision and time frameworks. It 
also serves the purpose of a concrete communication platform that facilitates scenario analysis, 
what-if alternatives and forward looking. We specifically define agent-based modeling within 
the larger simulation domain, provide a brief overview of other computation modeling meth- 
odologies and discuss the concepts of multiple models, verification, validation and calibration. 
The conceptual framework section closes with a discussion of advantages and disadvantages of 
using simulation modeling for policy at various stages of implementation. Finally, we present 
a panorama of actual applications of simulation modeling in policy, with an emphasis on eco- 
nomic analysis. 


Policy as a Complex System 


Policymakers face the arduous task of balancing interests, pressure, bureaucracy and evidence 
to foster a construed consensus of social betterment. In fact, more often than otherwise, there 
are disagreements about what the priorities are, which goals to seek, what methods are available 
and, if correctly and fully implemented, which endings will result (Gaus 2021). This descrip- 
tion shares a number of characteristics with a complex system definition (Furtado and Sakowski 
2014). 

Policies and their effects thereafter are obtained from the interaction of heterogeneous agents: 
citizens, sub-groups, bureaucrats, interested parties, lobbyists, activists, leaders and their contex- 
tual environment. These interactions among different actors produce emerging results embed- 
ded with non-linearities that diffuse across scales in time and space with distinct levels of rigidity 
and friction. Moreover, those who benefit and those who suffer negative externalities react. 
Together, such systems evolve, adapt, change and learn. 

This understanding that policymaking may involves complex systems implies that they are 
hard to predict (Mitchell 2011; Polhill et al. 2021). Furthermore, full understanding of a single 
mechanism does not guarantee the comprehension of the whole: “the whole becomes not only 
more but very different from the sum of its parts” (Anderson 1972, 395). 
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Considering that policymaking operates and is intrinsically entangled with a social complex 
system suggest that a conceptual view of the system is needed and that appropriate methodolo- 
gies should be applied when handling policy interventions. Bettencourt (2015) illustrates how 
some policy problems may accept linear, additive solutions whereas others, classified as wicked 
or complex, may not. Traffic flow through an intersection is a problem that specific equations 
and variables that account for flow and speed might lead toward an optimal and unique solution. 
Urban mobility and social violence, however, are phenomena that involve multiple actors and 
themes for which there may be not a single solution. Instead, a pathway of possible alternatives, 
at times contradictory, might or might not be achieved, depending, for example, on how parties 
engage, react, contribute or participate. 

This scenario that policymakers face makes simulation in general and agent-based modeling 
in particular suitable tools. Even more so when considering the growing availability of data, 
computational power and the need for interdisciplinary teams (Polhill et al. 2021), because 
simulation provides teams with a platform for communication, among other things: a common 
language that functions as a repository for concepts, understandings and behaviors that neces- 
sarily will have to function together to produce results. Moreover, in solving explicitly designed 
problems systemically, iteratively, over trial and errors, it helps foster communication among 
participants. Gilbert et al. (2018, 1) argue “the main benefit of designing and using a model is 
that it provides an understanding of the policy domain, rather than the numbers it generates”. 

Simulation of policy interventions provides two further advantages besides enabling a com- 
mon platform for communication. Simulation focuses on prognosis, analyzing future pathways 
and doing so via comparison with nonexistent alternatives, the so-called counterfactuals. Not 
being able to use experiments hinders social sciences when compared with natural sciences. A 
health policy analyst cannot ‘test’ a vaccination policy and evaluate ex post whether the decision 
was positive or not. Even before policy, pilots are tried out, or randomized-control trials are 
applied (Gilbert et al. 2018); ethical warrants are needed to insure no harm comes to citizens or 
to the environment from either the policy intervention or its omission. 

As simulations happen within computational environments, on top of artificial societies of 
their own (J. M. Epstein and Axtell 1996), they provide enough room for social scientists to 
experiment and test alternatives in a safe space. These experiments — these simulations — serve to 
evaluate possible scenarios, to explain broad consequences and to delineate the space of possibili- 
ties. As a result, simulations contribute by anticipating outcomes of policy actions. 

Epstein and Axtell (1996) proposed these artificial societies along with the generative con- 
cept for social sciences. Their motto was later set as ‘If you didn’t grow it, you didn’t explain it’ 
(Joshua M. Epstein 2007, 8). 

The context to which a simulation tool may apply is also relevant (Edmonds et al. 2019; 
Edmonds and Meyer 2017). Depending on the purpose of the model and the stakeholders 
involved, different validation expectations arise. Models that claim prediction, for instance, 
should provide results that match out-of-sample data: anticipating data that is unknown to the 
modelers and has not been used in the model itself. 

Edmonds et al. (2019) suggest seven purposes for simulation models, from the more specific 
to the more general: prediction, explanation, description, theoretical exploration, illustration, 
analogy and social learning. When it comes to the most abstract, social learning, the aim of 
the model is solely to function as a mediator, a platform that members of a group use to share 
information, concepts and meaning about a common problem. Therefore, no anticipation of 
accurate data is demanded for the model to achieve its set goal. 

In a broader context, simulations are models, abstractions of complex phenomena that mod- 
elers run to gain insights and systematic outputs. Models suffer criticism as they are based on 
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assumptions, theories and simplifications. However, in general, they are considered useful (Box 
and Draper 2007). Simulations do not demand exclusiveness on the policy phenomena analysis. 
Quite the contrary: there is evidence that multiple models are complimentary and may together 
encompass a more comprehensive understanding (Page 2012). 

Besides this introduction, the next section defines agent-based modeling and section 2 pres- 
ents some applications of ABM for policy. 


Agent-Based Modeling (ABM) as a Policy Tool 


Poledna et al. (2020) claim that Enrico Fermi made the first use of the agent-based model 
(ABM). In the 1930s, Fermi analyzed neuron transport by hand. Orcutt implemented an 
economic model in the late 1950s. However, the famous first model is considered to be 
the racial segregation analysis of neighborhoods made by Schelling (1969, 1972). The first 
attempt to make a systematic construct of concepts and inaugurate social modeling could be 
pinned to the seminal book by Epstein and Axtell (1996). In a similar pathway in economics, 
Tesfatsion and Judd (2006) coined the foundational agent-based computational economics 
(ACE). 

A general formulation of the concept of an agent-based model used to describe an artificial 
society (J. M. Epstein and Axtell 1996) is: 


Am = f(A,,E,) 
En = g(A,E,) 


which is a “discrete time dynamical system” (J. M. Epstein and Axtell 1996, 19) in which the 
states of agents and the environment in time f+1 depend on agents and the environment in t. 
How the transformations occur depend on the rules that are codified as f(.) and g(.). In practice, 
a simulation attempts to apply theoretical and tacit rules to data and “animate” (Galan et al. 
2009) the agents in time and space to obtain patterns that are empirically observed. 

When defined in contrast with models in which the “whole population . . . is collapsed into a 
single set of variables, [ABM can be defined as] computer simulations where individual elements 
of the social system are represented as separate elements in the simulation model” (Edmonds and 
Meyer 2017, 3). 

These definitions of ABM encompass its most useful features and delineate where their appli- 
cation is fruitful. Given the emphasis on the heterogeneity of agents, phenomena that include a 
number of different agents interacting in a decentralized way (policy arenas, for example) may 
be a suitable use. Moreover, if those agents are autonomous and active, in the sense that they 
learn, evolve, adapt and react depending on their contextual situation, considering their intrinsic 
nature but also elements of the environment (policy arenas, again), then ABM may be adequate. 
Finally, computational simulations are fit to include space in detail (Hammond 2015). Doing the 
same in equation-based models would make them quickly intractable. Although using deter- 
ministic computational code, simulations need not impose monotonicity and convexity (nor 
equilibrium) in order to be computed. 

Note that the definition of ABM and Epstein’s proposed motto (2007) include a necessary 
condition to understand a phenomena: being able to grow it — that is, replicate the mechanisms 
needed for the creation of patterns — although it may not be a sufficient one. The simulated 
model produced is but a candidate at explanation. There might be models that are unknown 
to the modeler that would serve the purpose of growing recognizable patterns. Automated 
processes, operated via machine learning, for instance, could produce other models that are 


273 


Bernardo Alves Furtado 


efficient in producing predictions. ABM, however, focuses on agents’ cognition and space-time 
descriptions that encompass reasonable parameters. “To explain a social pattern, one must show 
how the pattern could emerge on time scales of interest to humans in a population of cognitively 
plausible agents” (Joshua M. Epstein 2006, 1587). 

The definition of complex adaptive systems (Holland 1992; Furtado and Sakowski 2014) 
intrinsically implies that there is not just a single possible future trajectory for a system. Con- 
versely, a number of pathway alternatives may occur, depending on both the initial conditions 
of the system and the interactions among the system’s constituent parts. Although unpredictable, 
reasonable near-future states probabilities are possible, constrained by the assumptions made and 
the current status quo knowledge of the relevant mechanisms. In practice, scenarios — which 
alter main inputs and mechanisms of a modeled system in a controlled manner — contribute 
to aiding decision-making processes that take place independent of certain comprehension of 
future effects (Maier et al. 2016). 

Given this general conceptualization of agent-based modeling, we proceed to list some 
advantages and disadvantages of applying ABM to policy. Most of the advantages are bound to 
the definition of ABM. Agents, environment, interactions, space and time set a framework for 
analysis that systemically incorporate theory, data and tacit knowledge to provide computational 
experiments, counterfactuals and scenarios. The disadvantages depend on the specification of 
each one of those items, on the construction and implementation of the model and on its actual 
use for decision-making. 

Some of the advantages come from the contrast of typical equation-based methods. ABM 
proposes cognitive plausible agents (Joshua M. Epstein 2007) with limited rationality (Arthur 
1994), instead of working on aggregate measures (Edmonds and Meyer 2017) and agents that 
detain complete knowledge of the present and future in infinite lifetime style (Fagiolo and 
Roventini 2017). ABM uses an empirical spatially explicit environment (Taillandier et al. 2019), 
not one into which space is difficult to incorporate, except in a highly stylized way (Thisse et al. 
2021). Note, however, that instead of opposites, ABM and equation-based methods may be 
complementary (Grabner et al. 2017; Beaussier et al. 2019). 

In addition to those structural advantages, ABM has been shown to be useful to compare 
methodologies from distant disciplines (Chattoe-Brown 2021). The need for a formal (compu- 
tational) layout serves the purpose of laying the groundwork, which accepts additional modular 
levels and functions as a repository. Thus, ABM may help organize the multitude of inputs from 
different actors, interests, disciplines, methods and data. “A rise in interdisciplinary teams work- 
ing together to address pressing social challenges, leveraging the explosive growth of available 
data and computational power” (Buyalskaya et al. 2021, 1). Actually, explicitly involving stake- 
holders and making them hands-on modelers “increases the likelihood that the model will be 
used and will be fit for purpose” (Gilbert et al. 2018, 1). 

Another relevant advantage of ABM is its relative low-cost and fast in-silico implementation. 
Policy decision-making is necessary even in a crisis situation in which information and under- 
standing are lacking. Quickly prototyping might help develop a general sense of the effects of 
radical interventions. 

Disadvantages also abound when using ABM. A structural and intrinsic difficulty is to find 
the “appropriate level of abstraction” (Gilbert et al. 2018, 11). Complex models may include 
more details than necessary to describe a phenomenon and thus obscure relationships so that 
stakeholders and policymakers cannot pinpoint which inputs affect the outputs and by what 
order of magnitude. Conversely, the model may be so simple that the main mechanism associ- 
ated with the problem at hand is not present, thus rendering the model useless or even hazardous 
(Aodha and Edmonds 2017). 
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Kurtz and Snowden (2003) suggest that ABM is less equipped to handle people and organi- 
zations when compared with ABM applied to the analysis of natural systems. Specifically, they 
mention that humans may not be contained within a single identity or role; rather, humans are 
themselves complex systems. Kurtz and Snowden also criticize the fact that people would follow 
predetermined rules as they have free will and might follow collective opinions. Finally, Kurtz 
and Snowden suggest that people are able to act simultaneously on local and global scales and in 
the entire spectrum in between. 

Aodha and Edmonds (2017) present some specific recommendations when using ABM for 
policy analysis. Models in general, and ABM in particular, are loaded with scientists’ points of 
view, background and disciplinary formation, and those influence modeling assumptions and 
constitute a limiting of theoretical spectacles. Modelers and model users should also understand 
the model limitations and make sure that the model has been tested and checked thoroughly 
(Edmonds and Gershenson 2015). When the model lands on policymakers’ desks, Aodha and 
Edmonds (2017) suggest that it is likely that no third party has audited nor tested the model. 
It is common — given the complexity of the model and the tacit knowledge of its details and 
intricacies — that only a handful of people can properly run the model. Occasionally, not even 
the modeler but only the computer analyst can fully comprehend the model’s computational 
implementation (Galan et al. 2009). Further misuse of the model for policy, according to Aodha 
and Edmonds (2017), includes the aforementioned confusion between a model’s purpose, its 
goal and the context in which it was designed to be effective. 

A final disadvantage we present is the lack of a standardization (Beaussier et al. 2019). The 
generality and flexibility of the method contribute to a relatively lack of benchmark cases 
with example models varying in detail, implementation, documentation and description. 
Since the 2010s, some of this last disadvantage has become less relevant. In economics, for 
example, a number of ABM practices have risen providing minimum standards for macro- 
economic models (Dawid and Gatti 2018). In the social sciences, a strong community has 
emerged consolidating the debate on epistemological and ontological aspects of the meth- 
odology (Edmonds and Meyer 2017). In terms of communicating the models, a standard 
protocol was proposed in 2006 (Grimm et al. 2006), and then it was adopted by the com- 
munity, extended and adapted in more recent years (Grimm et al. 2010, 2020). Hammond 
(2015) lists some best practices. 

The conceptualization of ABM is not complete without the discussion of verification, cali- 
bration and sensitivity analysis and validation. There is not absolute consensus in the literature, 
especially about validation (Galan et al. 2017). Thus, we present a brief overview of each term. 

Verification relates to the formal codification of the model and to errors of computation. 
This happens when the programmer believes the computer is calculating a precise division, for 
example, but given internal mechanisms of the language, what actually happens is different. That 
may occur also when the program has run from beginning to end, apparently without throwing 
an error, only because the error is not one of syntax but of interpretation: an update of a state 
that should have happened but did not. Galan et al. (2009) also define errors as artefacts, which 
is the incorrect interpretation of the model. Artefacts take place when the modeler presumes a 
given assumption or process is generating a specific result, when other assumptions or mecha- 
nisms are the real influence. 

Calibration is the process of iteratively (and often in an automated manner) adjusting empiri- 
cal data to make the model compatible with observed phenomena. Some caution is warranted: 
data used for calibrating a model should not be the same data used for validating the model. 
When doing calibration, the modeler searches for the best value for parameters so that the model 
replicates empirical patterns and stylized facts. 
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Sensitivity analysis is somewhat similar as different configuration runs of the model are com- 
pared among themselves. However, the emphasis here is that, given a calibrated, verified model, 
what are the effects of a change in specific parameters? At times, the analysis illuminates explana- 
tory mechanisms and conditional scenarios (Gilbert et al. 2018). As a result, sensitivity analysis 
may work as a test of robustness or as an exploratory tool. 

Validation — although central to the modeling process — is a concept whose definition is 
more ample (Ngo and See 2012; Moss 2008; van Vliet et al. 2016). Moss (2008) and Galan 
et al. (2017) advocate that validation be attached to the purpose of the model. Hence, a model 
is valid when it fulfills the goal that was originally set. Validation is also defined as the “accurate” 
replication of a time-series set of data (Guerini and Moneta 2017). A partial compromise is 
that different modeling purposes demand different validation processes (Edmonds et al. 2019). 
Models that aim at predicting data would need to have proved themselves to have replicated 
out-of-sample empirical data that has not entered in the model, either as input or as calibration. 
A model proposed as a participatory analogy among stakeholders, policymakers and scientists 
might still be valid and function as a systemic template for knowledgeable interactions and com- 
munication without necessarily replicating existing data. 

Independently of which validation definition a modeler uses, the more aspects of the model 
that are validated the better. Thus, replicating and validating agent behaviors’ via a survey while 
simultaneously producing aggregate validation of empirical indicators (Haldane and Turrell 
2019) and a higher number of them provides more validation to the model. 


Applications of ABM for Policy 


Discussion 


ABM was propagated as a “promising feature” for scientific and policy analysis in the 2000s and 
2010s (Jovanovic et al. 2012): a possible candidate to surpass current best practices (Farmer et al. 
2015) and a likely tool to integrate economic and ecological models (Lamperti et al. 2019). 
Mainly, the promise was to assess “the relative merits of alternative policy prescriptions in meet- 
ing the policy objectives” (Gilbert et al. 2018, 2, emphasis added). 

The promising methodology has evolved into a varied array of examples. Lee et al. (2015) list 
works in ecology, economics, health care, sociology, geography, anthropology, archaeology, bio- 
terrorism, business, education, medical research, military tactics, neuroscience, political science, 
urban development and land use and zoology. More discipline-specific reviews may be found 
in the area of geography and cities (Heppenstall et al. 2012; Batty 2018), policy (Furtado et al. 
2015; Edmonds and Meyer 2017) and economics (Dawid and Gatti 2018). Here we present just 
a few examples that actually succeeded in the policy-applied criteria. 

Our first example that tries to improve “efficacy of policy response” (Carley et al. 2006, 282) 
comes from a model that incorporated actions and interactions of agents within their social 
and professional networks. The departing point of the model is the fact that diffusion and out- 
break of diseases happen amidst people as they go about, incorporating aspects at the physical, 
biological, social and economic environment. Carley et al. develop BioWar to assess numeri- 
cally the impact of policy interventions. BioWar functions as an integrated platform in which 
modular models are inserted in specific time points in the simulation, generating independent 
parts that function sequentially. A larger scope model — the “simulator state machine” — starts 
the process and at a given moment calls an “agent state machine”, which, in turn, eventually 
runs a “disease state machine”. In fact, the agent model receives inputs from data but also from 
five other sub-models: (1) disease, (2) geography, (3) weather, (4) attack and (5) communication 
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and technology. The model is validated against data on absenteeism and medical information. 
The validation runs in an automated tool in order to handle the complexity of the model. The 
automation allows for both validation and gain of mechanism understanding made via specifi- 
cally formulated queries. 

Kerr et al. (2021) implement this same idea of using ABM to evaluate policy interventions 
in a pandemic state in a number of countries across the world. COVASIM implements COVID 
progression, from susceptible to recovery or death for each individual agent. Agents interact in 
contact networks at home, in their communities, and at their school or workplace. Genera- 
tion of synthetic agents, households, schools and workplaces comes from population data. The 
authors state that they verified the model thoroughly, presenting a number of embedded tests. 
Calibration was used to approximate parameters of the model to real-world data, whereas vali- 
dation was applied upon actual use on a policy decision. COVASIM (Kerr et al. 2021) provides 
an array of policy interventions, including (1) physical distancing, masks and hygiene; (2) testing 
and diagnosis; (3) contact tracing; (4) isolation of positives and quarantine; and (5) vaccines and 
treatments. Treatment tests include expected reduction of the probability of progression of the 
disease. Specifically, COVASIM tested mobility restrictions imposed after the run of the model. 
“Despite a rapid increase of cases in the preceding weeks, the model predicted counterintuitively 
that even these modest mobility restrictions would be sufficient to stop the rise in cases, a pro- 
jection that turned out to be accurate” (Kerr et al. 2021, 24). At the same time, a counterfactual 
policy intervention, that of not having imposed additional restrictions — for obvious reasons — 
could not be tested but was predicted to have generated a three-times-higher infection rate. 

Gilbert et al. (2018) provide some examples of models that have been successful in changing 
policy parameters and being deployed in other occasions afterwards, such as Silent Spread and 
Exodis-FMD, INFSO-SKIN, the abstractor behaviour model. Silent Spread was first applied in 
2003 following the foot-and-mouth disease outbreak of 2001. 


The modelling was critical to the Government’s decision to relax the 20-day move- 
ment control to 6 days, subject to commitments from the livestock industry. The itera- 
tive, participatory development process generated an unprecedented level of ‘buy-in’ to 
the results in an area which had previously been marked by deep controversy. 

(Gilbert et al. 2018, 9) 


INFSO-SKIN model, in turn, was key to helping design a funding-guideline policy before it 
was actually implemented (Ahrweiler et al. 2015). The model compares a no-change-in-policy 
baseline with four alternatives. The modelers tested changes in scope, instrument, amount of 
funding and incentives to smaller groups’ participation. In short, the model simulates a market 
in which agents of varying size and complexity try to sell innovation to other users, trying to 
adapt, learn, cooperate and change its performance in the process to that they can increment 
their knowledge base. Together, agents associate in research consortia to make proposals and seek 
funding from well-delineated call criteria. The baseline scenario follows empirical data obtained 
for the Framework Programme 7 from the European Commission, whereas the previous six 
frameworks were used for calibration of the model. Ahrweiler et al. (2015) tested ten specific 
experiments with varying degrees of recommendations. According to Gilbert et al. (2018), 
INFSO-SKIN results informed the next funding cycle. Paier et al. (2017) also apply ABM to 
knowledge creation and policy recommendations. 

ABM in economics is far from consensus or mainstream (Fagiolo and Roventini 2017). 
However, the field has provided a benchmark of good practices (Dawid and Gatti 2018) and 
illustrations of models central banks use to inform decisions. Mostly, the models emphasize 
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macroprudential policies to reduce real estate market volatility. In a very specific context and 
with a heavily empirically calibrated model, stakeholders investigate the sensitivity of loan-to- 
value parameters to bound price volatility. A model developed and extended for Washington, 
DC, was applied later in the United Kingdom and evaluated in Denmark (Geanakoplos et al. 
2012; Baptista et al. 2016; Goldstein 2017; Carstensen 2015). 

Apart from these specific real estate market applications, Dawid and Gatti (2018) list appli- 
cations on fiscal and monetary policy, financial regulation and crisis management, labor mar- 
ket and regional growth. The authors emphasize that applications have become more specific 
and detailed since 2013, comparatively to more generic recommendations that were the norm 
before. Moreover, validation and testing have become more rigorous with a wider array of 
methodological tools available. 

The Bank of England (Baptista et al. 2016) and the Organization for Economic Coopera- 
tion and Development (OECD 2020) seem to have been supporting the use of integrated novel 
methodology such as ABM to tackle systemic challenges in the economy. A working paper pro- 
duced by the New Approaches to Economic Challenges (NAEC) (Naumann-Woleske 2021) — 
an initiative within the OECD — produced a review of ABM models that have helped make 
policy recommendations including (1) innovation and industrial policy, (2) macroeconomic 
policy, (3) financial regulation, (4) wealth and income inequality, (5) labor market policy and 
automation and (6) sustainability and decarbonization. 

The European Commission (EC) has also funded the Complexity Research Initiative for 
Systemic Instabilities (CRISIS) (Klimek et al. 2015). A recent breakthrough in validation and 
prediction mentioned by the OECD is the full simulation of the Austrian economy, made by 
Poledna et al. (2020). 

Poledna, Miess and Hommes aim at a full-scale, data-rich simulation, including all sectors 
and accounts of the economy. In a flexible model, without imposing equilibrium and with 
agents who make decisions based on partial information and limited cognition, the authors 
make an explicit effort to compare it with typical vector autoregression (VAR) models and 
dynamic stochastic general equilibrium (DSGE) models. The authors claim that the model is 
able to compete with VAR and DSGE models with the bonus of providing a forecast of disag- 
gregated sectoral variables and components of GDP. This responds to a need posed by Farmer 
and Foley (2009). 


Final Considerations 


This chapter describes the use of the agent-based model as a scenario tool to aid policymakers 
anticipate effects of policy intervention. The text frames policy as a complex system and presents 
the concept, advantages and limitations of ABM as a tool. We present a few applications of ABM 
for policy that were useful. Our take is that ABM has come from a potential and useful method- 
ological tool in the 2000s to a real, effective one in the 2010s, albeit less so for policy analyses. “The 
methodological difficulty is to bridge the gap between policy practice, often expressed in qualita- 
tive and narrative terms, and the scientific realm of formal models” (Ahrweiler et al. 2015, 1). 

The listings of actual use of ABM to policy intervention, although reasonable, are not over- 
whelming. However, ABM’s contribution varies from enabling the understanding of triggers 
and circumstances of social unrest (Jovanovic et al. 2012) to helping the development of theo- 
retical policy approaches (Klein 2021) or specifically indicating which epidemic intervention 
should be implemented and when (Kerr et al. 2021). But one must keep in mind the purposes 
of the model (Edmonds et al. 2019), its limitations (Aodha and Edmonds 2017) and the quality 
of its validation, given its context (Galan et al. 2009). 
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POLICY EXPERIMENTS 
AND PILOTS AS TOOLS FOR 
DECISION-MAKING 


Sreeja Nair 


Following the policy formulation stage, policymakers face the task of choosing a course of 
action from among a few plausible policy alternatives. What tools can be deployed by poli- 
cymakers to make the decision to go ahead with a particular policy? This chapter focuses on 
policy experiments, particularly pilots, as a policy tool to aid such decision-making. Policy pilots 
are commonly deployed to test policies before their full-scale rollout. Other intended uses of 
pilots are to encourage variation in policy responses, policy innovations and reducing policy 
risks, owing to their small scale and experimental nature. This chapter will focus on the ability 
of pilots in aiding decisions to scale up of experimental policy initiatives. Factors that impede 
the widespread uptake of policy experiments and pilots as a decision-making tool, despite their 
theoretical promise, pertain to the politics of experimentation and challenges in replication of 
success observed at the experimental or pilot stage on scaling up. 


Piloting and Decision-Making Under Uncertainty 


As policymakers move from policy formulation to decision-making, they are faced with the task 
of choosing a course of action from among a few plausible alternatives to address the policy issue 
at hand. Decision-making is a challenging activity as policymakers often make these choices 
under uncertainty or an incomplete understanding of the context in which the policy will be 
implemented and how it might change in future and impact policy performance (Swanson and 
Bhadwal 2009). 

Policymakers face a spectrum of such uncertainty that moves from limited knowledge 
about the future to the deepest layer of uncertainty or total ignorance, sometimes expressed 
as ‘unknown unknowns’ (Kwakkel, Walker, and Marchau 2010). Decision-making is not only 
affected by imperfect information (i.e., inaccurate or no information about the future policy 
context) but also by ambiguity (i.e., multiple interpretations of and diverse perspectives on the 
available information) (Jones and Baumgartner 2005). Uncertainty can also hinder a smooth 
decision-making process and result in solutions that are either futile or counterproductive to the 
achievement of intended policy objectives (Deyle 1994). 

One approach for policymakers to move ahead with decision-making despite the imminent 
uncertainty is to launch policies as experiments and pilots before deciding whether or not to 
introduce these fully (Shipan and Volden 2012). The experimental feature of these strategies lies 
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in their endeavour to demonstrate their superiority in addressing a policy problem among other 
available alternative strategies under consideration (Gardner 1995). A policy experiment can 
be defined as “a temporary, controlled field-trial of a policy-relevant innovation that produces 
evidence for subsequent policy decisions” (Huitema et al. 2018). In another useful definition, 
policy experimentation has been characterized as an iterative process of “testing, piloting or 
demonstrating” specific policy designs to gauge their potential as solutions to specific policy 
problems (Van der Heijden 2015). 

Under conditions of uncertainty, policymaking can adapt to continue staying relevant to the 
changing policy context over time. Preparing for futures that appear to be very different from 
the current context can be challenging to acknowledge and accept, and experimentation could 
be a channel to not only introduce but also test some alternative strategies for alternative futures. 
Adaptive policymaking encourages experimenting with policy alternatives to identify improved 
policy strategies as new conditions emerge, altering the original context in which the previous 
policies came about (Walters 1992; Swanson et al. 2010). When the original policy context 
shifts, consequent changes to status quo policies may become imperative, which could further 
have larger societal implications. Experiments can help in transitions to explore “a range of pos- 
sible pathways for change” (Farrelly and Brown 2011). 

While the terms experiments and pilots are often used interchangeably in policy literature, in 
practice these may not feature in any particular sequence, and pilots may or may not be preceded 
by an observable experimental stage. Pilots, in fact, can be considered as an advanced form of 
experimentation in terms of being more formalized and “visible” in the policymaking process 
(Nair 2021). Pilots can help test the “applicability, feasibility and acceptability of innovations” in 
varying new contexts and help in expanding these beyond the experimental stage (Rondinelli 
1993). Pilot projects can be a mode of experimenting with new programs at a “controlled small 
scale” before introducing full-scale programs (Swanson and Bhadwal 2009; Weiss 1975). Com- 
pared to other evaluative methods, such as impact assessments, that are formally conducted in 
many countries, pilots display a higher autonomy in terms of both who uses them and for which 
purposes (Nair and Vreugdenhil 2020). 

The value of experimentation in enabling improved policy designs is evident in calls for 
policies to be routinely launched as experiments and pilots (Sovacool 2012) or to be in a con- 
stant mode of piloting (Tomar and Swanson 2009). A major contribution of experimentation 
as a decision-making tool comes in the form of generating policy-relevant knowledge such 
as piloting for early assessment of impacts of new initiatives such as subsidies and incentive 
programmes (Stromsdorfer 1985; Martin and Sanderson 2016; Bevir 2009; Sovacool 2012). In 
the development sector, experiments have been routinely used to evaluate alternative strate- 
gies to allocate resources to those that emerge as most feasible (Rondinelli 1993). Experiments 
have also regularly found recognition as a decision-making tool by virtue of their evidence- 
generation ability for sectors such as education, health care, the environment and social wel- 
fare, among others (Bennion 2011; Cloutier et al. 2015; Seyfang and Smith 2007; Hoffmann 
2011). 

In countries with multiple levels of governance, launching policies as pilots can be a way 
to hedge risks that can result from one-shot full-scale implementation. Sub-national govern- 
ments thus become venues for experimentation to pretest policies and also facilitate policy 
learning (Lee, Ma, and Zhou 2017). Piloting can also help unearth and potentially correct any 
operational challenges prior to national-level implementation of chosen policies (Harrar and Lee 
Bawden 1972). In this sense, piloting becomes an ex-ante decision-making tool to pretest future 
programmes and policies for their likely impacts, process of implementation and stakeholder 
acceptability (Nair and Vreugdenhil 2020; McFadgen 2012). 
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To better understand pilots as a decision-making tool, it is useful to record some efforts to 
classify pilots based on their intended purposes. Impact pilots, for example, evaluate the likely 
effects of new policies, measuring early outcomes compared to a control population. Process 
pilots, on the other hand, aim at assessing the delivery mechanism itself: for example, which 
policy strategy is cost effective (Jowell 2003). In a study from England, well known for routinely 
conducting policy pilots, (Ettelt, Mays, and Allen 2014) summarize four purposes of pilots. 
These include piloting for testing if the policy can achieve its intended objectives in cost-effec- 
tive ways, with outcomes evaluated through randomized control trials. 

A second form of pilot includes those that are designed to “pioneer” or be an early form 
of implementation of a new policy for scaling up to the national level and mainstreaming with 
national policies. Some pilots are also launched to demonstrate policy success in chosen pilot 
sites that are known to have the capacity for conducting it and be a model for replication else- 
where. Pilots can also be launched as “trailblazers” for continuous learning and development, 
specifically focusing on aspects of policy operationalization, understanding barriers to imple- 
mentation and improving the overall implementation process when the policy is eventually 
introduced. (For more on policy learning, see Chapter 30.) 

Similarly, (Vreugdenhil et al. 2010) identify three types of pilots based on their purposes. 
These include, firstly, research pilots that focus on knowledge generation, either to test inno- 
vations and improve or evaluate policies that have already been chosen to be implemented. 
Secondly, management pilots that can be used to initiate communication and dialogue between 
multiple stakeholders in a bid to solve existing policy issues when other tools are inadequate or 
there is no consensus on them. Thirdly, political-entrepreneurial pilots that can be used to influ- 
ence policymaking for personal or strategic reasons. In another attempt to classify experiments 
more broadly (McFadgen and Huitema 2017) classify these into three types. Firstly, technocratic 
experiments, such as those focused on “fact finding” through generating knowledge while being 
shielded from political interference. Secondly, boundary experiments, in which policy solutions 
are developed considering multiple stakeholder value propositions. Thirdly, advocacy experi- 
ments for generating evidence to support policy choices that have been predetermined. 

These classifications themselves give an initial glimpse into the political nature of pilots and 
experiments in terms of the intentionality in launching them as well as directing the knowledge 
generated by them and utilization of the same for improving or reaffirming policy choices that 
may or may not be fixed at the time of launching these experiments. Despite their theoretical 
promise, pilots, just like full policies, are influenced by political factors. Factors that impede the 
widespread uptake of policy experiments and pilots as a decision-making tool despite their theo- 
retical promise pertain to the politics of experimentation and challenges in replication of success 
observed at the experimental or pilot stage on scaling up and form the focus of this chapter. 

The structure of the chapter in the following sections covers the decision outcomes that 
experimentation and piloting could lead to and a discussion of the benefits and drawbacks of 
using pilots as a decision tool. The chapter concludes with recommendations for effectively 
using piloting as a decision-making tool and suggestions for future research in this direction. 


Outcomes of Experiments and Pilots for Decision-Making 


Policy experiments and pilots can provide a variety of outcomes that can aid decision-making. 
In some cases, the outcomes can be intentionally driven while in other instances, these could be 
emerge in a less-controlled organic manner in the process of experimentation. In general, the 
motive behind launching policies as experiments is that demonstration of success in the experi- 
mental stage can open a path for these to be considered as full policies. Policymakers can also use 
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this opportunity to increase the acceptability of their preferred choices among the citizens in a 
bid to minimize any reputational risks for themselves arising from these choices (Peters 1998). 
(For more on behavioural tools in public policy, see Chapter 47.) 

A widely accepted outcome and indicator of success of a pilot is its geographical spread or 
scaling up to all target groups and beneficiaries (Hartmann and Linn 2007; Gillespie 2004). The 
heavy focus of policy literature on measuring the success or failure of pilots in terms of whether 
they scaled up or not has meant that less attention has been given to the range of types and 
forms of pilots that could survive in between these extremes and still continue to contribute 
towards improved policy designs (Nair 2021). Even without successful scaling up, pilots can 
generate lessons for the future and act as a foundation for systemic changes that could be social, 
cultural, political, institutional and behavioural (van Winden and van den Buuse 2017). This 
especially holds true when transitions to alternative futures are being discussed and considered 
in the policy arena. Even for unsuccessful pilots, lessons might be the biggest contribution 
(Mattingly 2008). 

Scaling up can be considered as expansion, replication, adaptation and sustaining of the pilot 
over space and time to reach a larger number of beneficiaries (Hartmann and Linn 2007; van 
Doren et al. 2018; Uvin 1995). Apart from increasing in size and reach, scaling up can also be 
considered as increasing the impact of the pilot in terms of achieving its objectives (Bloom and 
Ainsworth 2010). While piloting at the micro scale has received much attention from govern- 
ments and non-governmental agencies alike, piloting and scaling up pilots and experiments to 
benefit national-scale programmes has not received as much attention (Simmons, Fajans, and 
Ghiron 2007). 

In a counter-process to scaling up is what Wellstead et al. (2016) term scaling down. The 
ambitions of experimental initiatives for scaling down are similar to those of scaling up in 
terms of testing processes and outputs that combine good practices, but for implementation of 
centrally driven policies to reach the local level with sensitivity to local conditions and broader 
policy learning. Like pilots launched at the small scale, those launched at the national level also 
face challenges in translating broad policy goals into localized policies while engaging stakehold- 
ers. This phenomenon of scaling down, while important, is less studied. For decisions pertaining 
to action at a national level, such as climate change and pollution control, testing of ideas that 
indicate a broad strategic direction and principles is needed (Wellstead et al. 2016). 

Another important yet less studied outcome of pilots is in terms of their policy translation. 
Vreugdenhil, Taljaard, and Slinger (2012) termed this outcome a pilot’s diffusion involving repli- 
cation and scaling up of the pilot whereby the qualitative and quantitative nature of the problem, 
actors, policy components and administrative layers, scope and, thereby, complexity of the pilot 
changes. Considering diffusion as a spectrum, at one end, the outcome of the pilot can move 
from no replication to initiation of new pilot projects or management projects or scale up into 
an expanded pilot temporarily or management project. At the other extreme, a pilot can dif- 
fuse to be institutionalized into new or ongoing policies, an indicator of full policy translation. 
The empirical evidence on the design features of such pilots and the process of their diffusion 
is, however, lacking (Vreugdenhil et al. 2010). As the pilot scales up with the addition of more 
elements, the original pilot may cease to exist (van Winden and van den Buuse 2017). 

Such scaling up and diffusion can be led by governments, also referred to as centralized scal- 
ing up (Appadurai et al. 2015), wherein the national (union) government is the main agent driv- 
ing the process. Centre-driven pilots, especially in a federal governance structure, are important 
vehicles to translate broad national-level policy decisions for implementation at the sub-national 
levels. Similarly, the uptake of a pilot (van Buuren et al. 2018) refers to the process of embedding 
and implementing the results of the pilot within the current policy regime. 
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In the process of exploring policy alternatives to replace or supplement a current policy port- 
folio, policy pilots can aid the development of new policy designs or help build on earlier designs 
(Howlett and Rayner 2013). (For more on policy mixes, see Chapter 37.) 

As a decision-making tool, while some pilots might open the possibility of radical policy 
change, others may be rather incremental, with their primary contribution being in suggesting 
marginal adjustments to existing policies and programmes (Majone 1991). Questions of how 
much adjustment in policy response is required to match any changes in the future policy con- 
text and how best to do so remain a challenge. In this line, Nair (2019) draws attention to the 
aspect of proportionality of policy response and outcome in uncertain policy contexts, using the 
example of climate change adaptation decision-making. Adaptation strategies to a changing cli- 
mate, for example, could involve testing decisions that move from incremental to transformative 
policy strategies. These strategies could be tested through pilots as models of changes to status 
quo policy responses in dealing with different levels of change in the policy-making context: for 
example, changes in climatic variables. 

Institutionalization brings the innovative ideas of a policy experiment or pilot to receive 
widespread attention and can trigger necessary resource allocations and stability in terms of 
support for its implementation. By entering formal decision-making and implementation, how- 
ever, the pilot could lose its original flexible design and become standardized (Frantzeskaki 
et al. 2010). Lack of institutionalization of an experiment or pilot is also a likely outcome of 
the decision-making process and indicates that it has not been mainstreamed as a policy option 
(Vreugdenhil, Taljaard, and Slinger 2012). Even after the launch of an experiment or pilot, 
corrective iterations can feed back into the original experimental or pilot design continuously, 
especially when the experiment or pilot was found to be a failure in practice or was not insti- 
tutionalized (Wilder and Howlett 2014). Lack of institutionalization, however, does not neces- 
sarily always indicate failure of the pilot. Instead, it suggests that there were other ways in which 
the pilot and its learning could have been assimilated into ongoing policies. 


Experiments as a Decision-Making Tool: Benefits and Drawbacks 


Policy experimentation and piloting are far from being straightforward technical tools to yield 
effective decisions. Policy experiments and pilots are not insulated from the political aspects 
of policymaking, just as in the case of full policies. The push and pull between the technical 
merit of a policy experiment or pilot and its political acceptability and feasibility determine 
its usefulness at the decision-making stage. Spicer et al. (2014) argue that “scaling up is a craft 
not a science”, referring to its predominant political nature compared to its technical aspects. 
Despite impending uncertainty, the strategies that are selected might be those that are politically 
feasible rather than scientifically sound, especially for wicked problems such as climate change 
(Knaggard 2014). In the past decade, scholars have also considered focusing on the process of 
experimental policy design, including the role of various stakeholders therein, as compared 
to the earlier works that focused more on the content of the experiments themselves (van der 
Heijden 2013). During this period, the focus has also been on “experimentalist governance” as 
an iterative process of “provisional goal setting” for revising policy goals based on the learning 
from testing alternate modes of goal achievement in different contexts (Sabel and Zeitlin 2012). 

For politicians and bureaucrats, piloting may also be an opportunity to initiate policy change 
and gain recognition through demonstration of a policy idea. For policy researchers, on the 
other hand, pilots provide a chance for rigorous experimentation with policy design with the 
intention of establishing whether the proposed policy strategy/strategies are more effective than 
incumbent or previous ones. These purposes can co-exist, often unknowingly, which could 
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cause issues in determining the aims and objectives of any evaluation of these pilots (Ettelt and 
Mays 2015). 

Experiments and pilots can be launched with the idea of generating policy learning. In most 
cases, they provide direct learning outcomes after being launched purposefully by policymak- 
ers or indirect learning wherein the results of the pilots ultimately inform decisions on policy 
options (McFadgen and Huitema 2017). Ettelt and Mays (2015) consider policy pilots to be 
time-bound policies that are implemented over a limited spatial scale for the purpose of policy 
learning in some form. This can include piloting schemes that are intended to be “trailblazers”’, 
“demonstrators”, “pioneers”, “vanguards” and “early adopters” (Ettelt and Mays 2015). This 
coincides with the idea that policymakers have a stake in a policy direction, even while experi- 
menting. Experiments are thus an investment of political capital, just as with any full policy 
(Campbell 1991). 

While the contribution of experiments and pilots to decision-making is acknowledged in 
theory, in practice, governments might hesitate to deploy them regularly for the fear of appear- 
ing to be in a constant indecisive mode of experimentation (Peters 1998). In addition, despite 
their promising potential, policymakers might be hesitant to invest in experiments and pilots 
owing to the associated risk of failure. Additionally, to safeguard themselves from accusations of 
error of judgment and hesitance in acknowledging impending uncertainty about the effective- 
ness of chosen strategies, policy experiments can rarely be seen or accepted as a failure (Howlett 
2012). The underlying motive of decision-making and searching for effective solutions to a 
policy problem thus involves discovering not only which actions are technically capable of solv- 
ing a problem but also which among these is politically acceptable and administratively feasible 
(Howlett, Ramesh, and Perl 2009). Policymakers are also conscious of being accused of over- 
looking equity and fairness concerns by investing resources brought by experiments and pilots 
for certain sections of the society only (Stoker 2010). Participation and exclusion from these 
experiments is thus also political (Chien 2019). 

As those in government are conscious of their reputations via the performance of policies 
during their terms in office, they tread the path of experimentation precariously, even when it 
is called for and especially when experiments or pilots are likely to bring a major change to the 
status quo policies (Majumdar and Mukand 2004). Through planned outputs from “pre-emptive 
policy experimentation”, some policymakers could attempt to extend their influence and seek 
longevity for their ideas and policy decisions beyond their official terms (Callander and Hummel 
2014). In the context of climate initiatives and politics around decarbonization, Bernstein and 
Hoffmann (2018) caution that experiments may become a reason to lock in some favourable 
policies. Given the transient nature of politicians and the according shifting of priorities, the 
results of experiments may not obtain traction and support if these appear after there are changes 
to the incumbent policy regime that initiated these experiments in the first place (Huitema et al. 
2018). 

Considering experiments and pilots as a useful decision tool is also an outcome of knowledge 
on a subject and a policymaker’s awareness of the same. For example, even when knowledge is 
available on a subject, policymakers may not be aware of it; undertaking the decision-making 
in this case operates on an “uniformed ignorance” basis rather than informed awareness. In case 
knowledge on an issue is generally lacking, decision-makers may be aware of this gap and func- 
tion with “prudent awareness”. Or, if they are unaware of their ignorance, they could assume a 
“hubristic attitude” or overconfidence (Becker and Brownson 1964). 

Another challenge for using experiments as an effective decision-making tool is in terms 
of whether these are used in a haphazard and ad hoc manner or form a conscious and routine 
activity on the part of governments. Governments often experiment in a trial-and-error manner 
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with little knowledge cumulation followed by policy improvement. This could be ascribed to 
the movement of bureaucrats across administrations and changes in parties in power (Pyle 1980; 
Peters 1998). 

The outcomes of a pilot can also be tampered with or affected by political influences. For 
example, pilots that are designed to investigate gaps in ongoing policies cannot serve this pur- 
pose if errors or gaps identified in the pilot phase are not corrected before the decision is imple- 
mented fully (K’Akumu and Appida 2006). Piloting is typically undertaken to allow for formal 
policy learning through its evaluation, but in practice, not all pilots are formally evaluated (Ettelt, 
Mays, and Allen 2014). In another study in England, Martin and Sanderson (2016) observed that 
pilots were used to identify “trail-blazing local authorities” and share good practices rather than 
to inform policy change. In this sense, the pilots became an exercise in early implementation 
rather than an opportunity to review the appropriateness of the policy. 

Resistance to change from the status quo, even when the results of a policy experiment or 
pilot call for it, can be difficult owing to the “stickiness” of policies. Policies which might be 
implemented wrongly are one disaster, but the chances of the impacts of these wrong decisions 
being significant is a bigger one (Tassey 2013). Continuation and expansion of policy pilots can 
also be stalled if there is widespread opposition from key stakeholders (Vreugdenhil et al. 2010). 
Political pressure can also hasten the process of evaluation of pilots to showcase evidentiary 
support for the implementation of certain decisions that were the preferred choices all along 
(Sanderson 2002; van Buuren and Loorbach 2009). 


Concluding Insights for Policy Theory and Practice 


Policy experiments and pilots can be useful decision-tools in re-evaluating or reaffirming 
policy choices that emerge after the policy formulation stage. However, how can policymak- 
ers strengthen their role as an effective decision-making tool while avoiding the traps that 
are associated with experiments and pilots, largely owing to the inherent political nature of 
experimentation? 

Successful design, implementation and scaling up of experiments are not automatic processes 
(Nair 2021). Table 23.1 summarizes some of the issues that affect the use of experimentation 
as a tool for policymaking under uncertainty, presenting an interplay of meaning and power. 
While “meaning’’-related issues relate to design of experiments considering how and why the 
future is interpreted in a certain way, issues related to “power” deal with how and why subse- 
quent courses of action are chosen. There are issues of politics surrounding experimentation 
and the presence of diverse stakeholder perspectives. Thus, the launching of experiments and 
pilots should be accompanied by careful thought and deliberation, just as in the case of full- 
scale policies. While issues of “meaning” can be better addressed by enhancing the knowledge base 
for informed policymaking, issues of “power” need higher levels of engagement and dialogue 
between stakeholders to agree on suitable courses of policy action for the future, based on the 
insights gained from well-crafted policy experiments (Nair and Howlett 2016). 

Given the political challenges attributed to scaling up and wider and sustained uptake of 
results from experiments and pilots, it appears that experiments are more welcome in forms 
that are small, exploratory and with modest aspirations and as tools to indicate the need for 
policy change without effecting change themselves (Huitema et al. 2018; Boeckmann 1976). 
The conditions that mark the freedom of a pilot to experiment and learn also become barriers 
to its wider uptake, geographically as well as in terms of changes in policy goals and means (van 
Buuren et al. 2018) (van Buuren and Loorbach 2009). An implicit risk in attempting to replicate 
the results of pilots to larger scale is that the key features of the pilot might be lost (Simmons, 
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Table 23.1 “Meaning”- and “Power”-Related Challenges for Using Experimentation as a Tool for 


Policymaking Under Uncertainty 


Challenges 


Issues of “Meaning” 


Issues of “Power” 


Politics of 
policy 
experiments 


Evaluation 
of policy 
experiments 


Diffusion 
of policy 
experiments 


Where to experiment and why 


Difficulties in measuring and 
attributing causality and isolating 
policy outcomes that can be solely 
attributed to experiments 

How and when to conduct the 
evaluation 


Scanty and fragmented evidence on 
process of diffusion of experiments 
and influencing factors 

Design and results of experiments are 


Hesitation towards accepting uncertainty 
and risk of failure 

Hesitation in changing from policy status 
quo, especially when additional costs are 
involved 

Power struggle between central and local 
levels to control emergence of novel 
solutions via active experimentation 

Experiments launched to mask/delay 
politically unpalatable policy reforms 

Hastening evaluation to obtain “evidential 
support” for predetermined policy 
solutions 

Bias towards “prototyping” versus true 
experimentation to try what works 

Some experiments have long gestational 
periods before results become apparent, 
going beyond administrative or career 
timelines 

Discounting future policies against “less- 
than-optimal” policies choices today 

Diffusion of experiments influenced by 
multiple stakeholders 

Stakeholder attitude and support for scaling 
up can change over temporal and spatial 


often heavily bound by context scale 
specificity; thus, there are limitations 

to exact replicability of experimental 

outcomes when these are scaled up 


Source: Nair and Howlett (2016) 


Fajans, and Ghiron 2007). Additionally, some features that were proven to be successful at pilot 
scale could get deliberately chosen over others that may be ignored as pilots scale up. 

In terms of research on policy experiments and pilots as effective decision tools, studying the 
varied contributions that these could make to the policy process is essential, instead of view- 
ing success as all or none. Hoffmann (2011) argues that rather than studying individual pilots 
with context-specific limitations, it is more valuable to look at experiments as a collective. For 
example, studying the combined impact of independent initiatives for climate change action by 
cities, provinces and states, citizen groups and corporations globally could guide climate policy 
development in general. 

The scaling-up of policy experiments and their translation to full policies face several chal- 
lenges (Stoker 2010). This is more so if the findings of the experiments and pilots challenge or 
do not reaffirm predetermined or preferred policy choices of the government (Ettelt and Mays 
2015). The politics of experimentation could also mean a strong “intentional policy design” 
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with the central government governing both the content and process of experimentation at the 
local level, thereby stifling the rise of innovative solutions via active experimentation (Mei and 
Liu 2014). 

It is no surprise that all experiments, just like full policies, are political. All experiments 
essentially face a push and pull between technical and political factors and eventually influence 
their outcomes. In practice, one may see sub-optimal experiments getting scaled up and opti- 
mal ones getting stalled or terminated or not chosen at the end of the decision-making process. 
Despite its widespread acceptance as a prescribed activity for governments, experimentation is 
very political. That is, governments design experiments mindful of issues of reputation, protect- 
ing and prolonging their office term, projecting confidence in their decisions and launching 
experiments and pilots in a rather “‘safe manner”. 

In some cases, experimentation is initiated as a conscious activity with the intention of learn- 
ing from it while in others, it is a simpler effort to get it right the first time and maintain it if it 
is running well or is “good enough” at the minimum (Peters 1998). Making pilots work thus 
can involve entrepreneurship and creativity on the part of policymakers (Bailey et al. 2017). 
Protected spaces to allow experiments to run and for their results to emerge free from political 
influences could include strategic actions such as labelling a project as a pilot or demonstration 
(Farrelly and Brown 2011; Tassey 2013). Formal policy review also provides the space in which 
to pilot test policy instruments and learn lessons with regard to intended outcomes and efficient 
implementation (Tomar and Swanson 2009). Pilots should continue even after scaling up and 
diffusion as their short-time findings may or may not match their outcomes in the long term. 
The short-term results thus might even be misleading to base decisions on (Breckon 2015). 
Proper evaluation of experimental initiatives, free of political influences, is important (Hildén, 
Jordan, and Huitema 2017). In such an evaluation, the experimental process is subject to con- 
stant monitoring and adaptation to changing policy context, effectiveness and efficiency of the 
experimental design and is conducted in collaboration between those who design, implement 
and benefit from the experiment. 
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THE ORGANIZATIONAL 
TOOLS OF GOVERNMENT 


The “Forgotten Fundamental” 


Michael Howlett 


Organizational tools include a broad range of governing arrangements which rely on the gov- 
ernment’s ownership of and control over physical, fiscal and human personnel resources to 
produce desired goods and services to facilitate the achievement of policy goals. While a large 
variety of organizational tools is possible, all involve and rely on, to varying degrees, the use of 
government’s financial and personnel resources to perform their functions. 


Introduction: State Organizations as Policy Tools 


While policymakers can conceive and use hundreds of specific substantive policy tools, scholars 
have tried to facilitate their understanding by classifying them under a few broad categories. 
“Carrots, sticks and sermons”, for example, is a simple way to capture the essence of the substan- 
tive tool choices governments face (Bemelmans-Videc et al. 1997; Howlett et al. 2005). For the 
purposes of this chapter, the four-fold NATO schema proposed by Christopher Hood (1986) 
will suffice. This adds an organizational component to the three types of tools mentioned earlier: 
nodality (i.e., information or “sermons’”), authority (i.e., regulations or “sticks”’), treasure (i.e., 
economic incentives or subsidy “‘carrots”) and organization. 

All four types of tools are theoretically available to governments for executing their policy 
decisions, though some are more suitable for particular tasks than others. Thus, a government 
may choose to present information with the purpose of altering subjects’ behaviour, apply regu- 
lations to require or prohibit certain actions on the part of the subjects or use positive incentives 
(e.g., subsidies) to encourage a desired behaviour or negative incentives (e.g., taxes) to discour- 
age undesired behaviour. Finally, and especially important in the context of this chapter’s focus, 
governments may also choose an organizational form — e.g., government department, state- 
owned enterprise, or non-profit company — to produce desired goods and services and provide 
them to the populace. What it produces, under what conditions and the prices at which the 
products and services are sold have vital impact on societal welfare. 

The overall pattern in the use of policy tools over the past five decades, however, is one of 
the expanding reach of the market into the provision of public goods and services at all levels 
of society, including many which were previously supplied by government (Howlett 2011). But 
traditional command-and-control instruments of governance, such as regulation by government 
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or independent regulatory commissions, state-owned enterprises and direct taxation and subsi- 
dization, have continued to grow in many countries and sectors despite efforts at privatization 
and deregulation (Jordana and Levi-Faur 2004; Ramesh and Howlett 2006; Jayasuriya 2001, 
2004; Vogel and Kagan 2002). 


Types of State Organizational Tools: From Government Agencies 
and State-Owned Enterprises to Advisory Agencies 
and Administrative Tribunals 


Substantive Organizational Instruments 


There are many types of substantive instruments which rely for their effectiveness on the orga- 
nizational resources of governments and non-governmental agencies and actors. Most involve 
(and rely primarily) on the use of government personnel to achieve government goals, usually 
operating in structures created and controlled by governments. These are “direct” government 
organizations but can also include “indirect” or quasi- or parastatal ones; the best-known exam- 
ple of which is the state-owned or “public enterprise” — which itself comes in many shapes, 
sizes, colours and flavours (Bernier 2011) — as well as non-state ones, such as co-production or 
certification (Cashore 2002; Voorberg et al. 2015). 


Direct Government 


The direct use of government agencies for substantive policy purposes involves the “delivery of a 
good or service by government employees, funded by appropriations from government treasury” 
(Leman 1989, 2002). This is what has sometimes been referred to as “the forgotten fundamen- 
tal” (Leman 1989) within policy instrument studies as its ubiquitous nature is often ignored in 
studies focusing on more esoteric kinds of tools. 

Within this general type of direct government organizational tool, there are several common 
forms or subtypes found in many jurisdictions. These include the following: 


Line Departments 


In most countries, government agencies undertake a wide variety of tasks on a direct basis. 
These include, but are certainly not limited to, those listed in Table 24.1. 

These services are provided at all levels of government (central or federal, provincial, state 
or regional, as well as urban or local) in slightly different configurations in different countries. 
Unemployment, welfare and social security payments, for example, can be the task of central 
governments in some countries and eras and of provincial or local governments in others. 

Typically, modern government agencies follow what is known in the public administra- 
tion literature as a Weberian “monocratic bureaucracy” form of organization (Brubaker 1984; 
Beetham 1987). This is a type of organizational structure first systematically described and 
analyzed by the German political sociologist Max Weber in his early twentieth-century work 
Economy and Society, whereby: 


e Personnel are appointed on the basis of a merit system of appointment, retention and 
recruitment. 

e — Office holders do not own the office in which they work but hold it subject to the provi- 
sions of the merit system. 


298 


The Organizational Tools of Government 


e Offices tend to be organized in a hierarchical fashion with a relatively small span of control 
and multiple levels. 

e  Allactivities in the agency operate according to the rule of law — office holders are not above 
the law and must operate within its limits (including provisions for their accountability — 
via some form of “chain of accountability” — to representative assemblies in modern liberal 
democracies who actually establish and promulgate laws). 

(Albrow 1970; Weber 1978) 


What are commonly referred to as “line” departments (in order to distinguish them from cen- 
tral, headquarters or “‘staff” units) have this “classic” hierarchical Weberian monocratic bureau- 
cratic form and are thoroughly embedded in legal forms of governance. Such units are typically 
organized in a pyramidal shape, linking offices of civil servants in various branches and sections 
to a single department head, such as a department of health or a department of highways. A sub- 
variation of this is the “ministry”, a form in which, typically, multiple pyramids of departments 
culminate in a single head (e.g., a ministry of lands, parks and housing) or an “agency”, which 
operates separately from the policy-making level of managerial control (Verhoest et al. 2010). 

These forms of government organization are the “workhorses” of publicly provided goods 
and service delivery in most modern states (see Table 24.2 for an example of their use in govern- 
ment service delivery in a modern state, Canada). 

These kinds of organizations can be very large (the US Department of Defense, for example, 
has over two million employees, including approximately 650,000 civilians) and can be sub- 
divided into hundreds of separate branches, bureaus, sections and agencies. They employ the 


Table 24.1 Tasks Typically Undertaken by Government Agencies 


‘Task Examples 


Facilitating commerce 
Managing public lands 


Constructing public works 
Research, testing and statistics 
Law and justice 


Technical assistance, record 
keeping and libraries 

Health care 

Social services 

Education and training 


Labour relations 

Marketing 

Defence 

Supplying internal 
government needs 

Finance 

International affairs 


Mint, standards of weights and measures bureaus 

Commissioners of public lands, ministries of lands and parks or 
environment or natural resources 

Departments of public works — airports, highways 

National statistical agencies 

Courts, solicitor general or attorney general offices, corrections and 
prisons, policing 

Farm extension, ministries of agriculture, national archives, national 
libraries 

Ministries of health — hospitals, clinics, dentists, nursing, home care 

Ministries of welfare and social, family or community services 

Ministries of education, post-secondary education colleges and 
universities, technical and training institutes 

Ministries of labour and labour relations 


Tourism, ministries of small business, ministries of trade and commerce 


Ministries of defence, army, navy, air force, coast guard 
Ministries of supply and services, Queen’s printers 


Ministries of finance and treasury boards 
Ministries of external or foreign affairs 


Source: Hodgetts, J. E. 1973, The Canadian Public Service: A Physiology of Government 1867—1970, Toronto: 


University of Toronto Press 
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Table 24.2 Examples of Tasks Undertaken by Public Enterprises (Canada — Twentieth Century) 


‘Task Example 

Housing Canadian Mortgage and Housing 
Corporation 

Finance Bank of Canada, Small Business 


Wartime production 
Transportation 


Strategic industries 
Communications 
Cultural industries 


Utilities 
Infant industries 


Development Bank, Caisse de Depot et 

Placement de Quebec 

Canadian Arsenals 

Canadian National Railways, Via Rail, Air Canada/Trans-Canada Airlines, St. 
Lawrence Seaway Co., BC Ferries, Northern Transportation Company Ltd. 

Atomic Energy of Canada Ltd., Petro-Canada 

Canadian Broadcasting Corporation, Radio-Canada 

Canadian Film Development 

Corporation, National Film Board, National Museum Corps. 

SaskTel, Hydro Quebec, Ontario Hydro, BC Hydro 

Petrosar, Athabaska Tar Sands, Canadian Development Corporation 


Sick industries Skeena Cellulose, BCResources Investment Corporation 

British Columbia Building Corporation 

Prince Rupert Coal Corporation, DEVCO, Cape Breton Development Corp. 

BC Liquor Stores, Société des Alcools de Quebec, Casino Nova Scotia, 
Lotto-Canada 

Translink, Edmonton Telephones 

Canada Wheat Board, British Columbia Egg and Milk Marketing Board, 


Freshwater Fish Marketing Board 


Property management 
Regional development 
Lotteries and vice 


Local utilities 
Marketing boards 


Source: Vining, A. R., and R. Botterell. 1983, “An Overview of the Origins, Growth, Size, and Functions 
of Provincial Crown Corporations”, In Crown Corporations: The Calculus of Instrument Choice, ed. J. R. S. 
Pritchard. Toronto: Butterworths, 303—368 


most personnel and deliver by far the largest percentage of state-provided goods and services in 
liberal-democratic, and virtually all other, forms of modern government. 

The “government employees” employed in line departments are typically civil or public 
servants. In most liberal-democratic countries these are unionized and well-paid positions, and 
although this is not the case in many other countries where officials may supplement their wages 
illegally through various forms of corruption (“kickbacks”, bribes, “service” payments, expedit- 
ing “fees” and so on), in either case, the use of public servants to directly deliver public services 
is an expensive proposition, which in itself discourages its use. How well these officials are edu- 
cated and trained and what kinds of facilities and information they have to work with also affect 
their capacity and perceived competence and, along with cost, can pay a significant role in their 
placement within a policy design (Brunsson 2006). Countries or sectors with well-resourced 
administrative systems regarded as highly efficient and competent by their citizenry are more 
likely to feature direct government service provision in their policy designs than countries with 
corrupt or inefficient civil services, given the advantages the former often hold for governments 
in terms of cost and ease of program administration. 


Central Support Agencies 


These are agencies which are similar in appearance to line departments but often act more like 
private companies, delivering services within governments rather than to external constituencies. 
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Some of these are very old (like government stationers and printers) while others (like govern- 
ment systems and information technology units) are much more recent. Many of these agencies 
are quite large, and since they often serve functions similar to those of private companies, they 
are, and have often been, primary targets for government efforts to develop market modes of 
governance in some sectors through contracting out or privatizing government services — that 
is, they are simply turned into “firms” supplying government services by severing their funding 
through general appropriations revenue and establishing autonomous boards of directors. Cost 
issues are typically a major factor influencing their inclusion in policy designs. 


Savings and Wealth Funds: Social Security, Health Insurance 
and Pension Plans 


Social and health insurance and pension schemes like those used in many countries for unem- 
ployment insurance, elderly income support and health care are other such government organi- 
zation-based schemes, ones in which all individuals in certain categories are mandated to make 
payments with a government agency which acts, usually, as a monopoly insurance provider for 
that group (Katzman 1988; Moss 2002). Some of these schemes, of course, are among the larg- 
est areas of government expenditure and are virtually identical in organizational form to direct 
government organizational tools given their universal and mandated nature — with the main 
difference being that programme funds come from dedicated insurance payments rather than 
general tax revenue. These schemes are generally very high profile and targeted to specific kinds 
of outputs. They are often intended to be revenue neutral, although any short-term shortfalls in 
these schemes typically have to be made up by governments. They also can provide large pools 
of capital which governments can use to finance infrastructure and other kinds of investments. 
As a result, they are very popular and found throughout the world, although their configura- 
tion and extent of private sector involvement varies greatly from country to country. Countries 
which do not have such schemes typically cite reasons related to costs or intrusiveness in already 
existing private sector programmes. 


Quasi-Governmental Organizational Forms 


All these types of government agencies have an essentially bureaucratic organizational structure 
and also exist largely as Weberian forms of administration — although some, like pension and 
insurance funds, may be structured in a more “businesslike” fashion with fewer rules and regula- 
tions guiding their behaviour than government departments and agencies. 

Other kinds of organizational tools, however, are typically less bureaucratic in nature. These 
include the following main types. 


Public Enterprises and Other Corporate Forms 


Public enterprises or “state-owned enterprises” (SOEs) come in many different forms, depend- 
ing on how their share ownership is structured. 

Perry and Rainey (1988) developed an exhaustive typology by examining the different types 
of ownership, sources of funding and mode of social control exercised over these organizations. 
The key feature of these organizations, however, is that they have a corporate form and are 
not administrative agencies. That is, they operate under separate legislation or under general 
corporate legal principles, and government control is exercised indirectly as a function of gov- 
ernment share ownership, typically through voting control over appointments to the company 
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board of directors — who usually can be removed “at pleasure” by the government. The board 
of directors then hires and fires senior management so that government control is indirect 
and “arm’s-length”, unlike the management and control of direct government administrative 
agencies. 

Some public enterprises can raise and borrow money on their own authority while others are 
limited in their sphere of independence and must seek funding or permission to borrow from 
governments. Similarly, some are free to set whatever prices they would like for their products 
while others must seek government permission to alter prices and may be subsidized to provide a 
good or service at below market value. While government share ownership can drop below 50% 
and still exercise control if the remainder of the shares are widely held, it is more common for a 
government to own 50% or more of voting shares (in fact, it is very common for them to own 
100%). However, there are a growing number of “mixed” enterprises with joint public-private 
or multiple-government ownership. 

These companies can be exceedingly large, although they can also be much smaller, in some 
cases limited to one or two factories or offices. Sovereign wealth funds, holding the proceeds of 
oil and gas or pension revenues in countries like Singapore and Dubai, for example, are among 
the largest firms in terms of assets and can control hundreds of billions of dollars in investments 
(Elson 2008) while large public hydroelectrical or petrochemical utilities in countries like Can- 
ada, Norway, Mexico, Iran and Venezuela also rank first among companies in those countries 
based on size of assets controlled (Laux and Molot 1988). 

These efforts have been successful in sectors where competition exists, such as marketing 
boards, product-producing companies and property management (Savas 1987; Laux 1993) but 
have generally foundered in other areas where the privatized corporation has simply become a 
monopoly service provider. This has often been the case with large-scale utilities such as water, 
electricity or public transportation providers where natural monopoly conditions often exist. 
In these cases, they have often been re-nationalized or re-regulated through the creation of 
regulatory oversight agencies and mechanisms (Mees 2005; Leland and Smirnova 2009). Policy 
designers now very much take these contextual circumstances into account in proposing or 
recommending either the creation or privatization of SOEs. 

Table 24.2 provides examples of the many public enterprises used in the twentieth century 
in Canada, a country which has not been at the forefront of the deployment of such tools, but 
which has also not shied away from creating them when needed to pursue national and regional 
objectives. 


Organizational Hybrids 


In recent years, numerous hybrid forms of indirect government organizations have also been 
developed and implemented in many jurisdictions. These have often been proposed in situa- 
tions where governments would like to privatize or contract out government services but where 
there is not a competitive market, thereby limiting the utility of outright sale or divestment by a 
government (Mathur and Skelcher 2007). 

Examples of these types of tools include so-called “special operating agencies” (SOAs) (Kop- 
pell 2003; Birrell 2008), which were established in many countries in the 1980s and 1990s in 
an effort to grant more autonomy to central service agencies and remove them from day-to- 
day government control. This was typically done by “outsourcing” whatever services could be 
secured from a competitive external marketplace while allowing agencies providing those goods 
and services which could not be so relocated to charge real prices to purchasers and to retain 
their earnings and make their own reinvestment decisions (Aucoin 2006; Flumian et al. 2007). 
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A second type of hybrid is the “quasi-autonomous non-governmental organization”, or 
quango, an organizational form in which a non-governmental agency is established and given 
a grant of authority by a government to provide a particular good or service (Hood 1986). 
These can be precisely targeted and many airports, ports and harbours are run by such “inde- 
pendent authorities”, which rely on governments for their monopoly position but which are 
answerable to their own boards for their activities rather than to the government itself (Kickert 
2001). These agencies are usually then able to charge their own prices for the good or service 
they provide, retain their earnings and raise funds on capital markets for investments, removing 
these items from government books (Flinders and McConnel 1999; Lovink 1999; Advani and 
Borins 2001). 

There can be serious principle-agent problems with these kinds of agencies, however, which 
can affect their use in fields with legal or corporatist modes of governance (Koppell 2003). 
Maintaining the arm’s-length nature of the relationship of public enterprises and s to govern- 
ment is difficult, and such agencies may not have enough autonomy for governments to avoid 
the consequences of scandals or other problems associated with them. That is, these relationships 
can be either too close (day-to-day interference) or too distant (agencies become distant and 
aloof powers unto themselves). 


Partnerships, Commissioning and Contracting Out 


More recent efforts on the part of some governments to shift some sectors from legal or cor- 
poratist to market modes of governance — that is, to offload legal and financial responsibility for 
goods and service delivery and have existing goods and services delivered through the private or 
quasi-governmental sector — have evolved into several distinct forms of organization which are 
more private than public, with the public sector acting mainly as a purchaser of goods and ser- 
vices provided by private companies (Grimshaw et al. 2001; Grimsey and Lewis 2004; Greener 
2006). 

One typical form of such activity 1s “contracting out” or outsourcing, in which internal 
provision of some good or service is simply replaced with a source external to government 
(Zarco-Jasso 2005). This can be more complicated if a non-governmental provider does not 
exist for a particular product or service, so a government must first, or simultaneously, create a 
non-governmental provider (Brown et al. 2008). Outsourcing of highway and railway mainte- 
nance in many countries in the 1980s and 1990s, for example, involved government managers 
creating their own firms, which then bid on and received government contracts to provide 
maintenance services; those companies then immediately hired former government workers 
and, in some cases, used former government equipment to provide the same service (McDavid 
and Clemens 1995). 

Different kinds of these partnerships exist, such as collaborative partnerships with NGOs to 
control hospital admissions for the disabled, operational partnerships with companies and other 
governments to share costs for many of the items discussed in this chapter and contributory 
partnerships in which governments may provide funding without necessarily controlling the 
use of such funds, as occurs when matching funds are provided for local or community-based 
environmental improvement projects (Hodge and Greve 2007). 

Although popular in some countries and sectors in recent years as examples of “collaborative” 
or “joined-up” government, such schemes often stretch the resources of non-profit or volunteer 
organizations and can result in inefficient or incompetent goods and services delivery (Evans 
et al. 2005; Riccucci and Meyers 2008). These kinds of cost and reliability issues increasingly 
have affected considerations of such tools and their inclusion in policy designs. 
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Commissioning 


Commissioning is the most recently recognized collaborative technique in which, as Taylor and 
Migone (2017) put it: 


Generally refers to a more strategic and dynamic approach to public service design and 
delivery with a clear focus on aligning resources to desired outcomes and by injecting 
greater diversity and competition into the public service economy. By creating public 
service markets, the expertise and resources of the private and not-for-profit sectors 
can be harnessed and leveraged through new business and delivery models. 


Commissioning goes well beyond traditional procurement and outsourcing agendas with the 
aim of increasing service levels from both private and community partners by involving “third 
sector” actors such as NGOs in both service target formulation and design (“co-design”’) as well 
as service delivery (“co-management”). The provision of stable funding and ongoing interac- 
tions with government funding agencies, it is argued, allows the capacity of third sector actors to 
be enhanced at the same time that co-design and co-management ensure that outcomes match 
the expectations of clients rather than agencies. However, in practice, such efforts have also often 
run into challenges due to a mismatch between government and NGO capacities and resources 
(Bovaird et al. 2014). 


Contracting 


Contracting involves governments in reducing the level of direct state involvement in the pro- 
vision of public services, including internal state services, through the replacement of civil 
servants and internal procurement processes with contractual arrangements with (usually) non- 
governmental organizations, primarily businesses (Ascher 1987; Vincent Jones 2006). Various 
forms of contracting out exist, such as public-private partnerships (PPPs), which have grown in 
popularity in areas involving large-scale infrastructure investments, whereby private firms absorb 
many investment costs and construct or rehabilitate infrastructure projects in water, highways, 
dams and other such areas in exchange for long-term contracts to operate and profit from them 
(English and Skellern 2005; Vining et al. 2005). Other less rigorous forms of contracting out 
exist in areas such as health care, education and prisons, in which contracts may not involve 
large investments and profit sharing but rather simply remove major expenditure areas, such 
as penitentiaries, from government books and budgets (Thadani 2014; Roehrich et al. 2014). 

In the case of contracting, many supporters of the concept have noted significant limitations 
which can prevent contracting from functioning effectively. The 2016 Nobel Prize in Econom- 
ics, for example, was awarded to two economists who specialize in detailing the significant flaws 
and limits of contracting in areas such as prisons and health care (Holmstrom and Milgrom 
1991). Their concern was with the difficulty encountered by governments in enforcing quality 
control in such contracts when the nature of the service provided (i.e., its quality) was dependent 
on difficult-to-monitor interactions between, for example, patients and doctors or prisoners and 
prison guards. However, many other criticisms of public-private partnerships exist (Newman 
2014) which highlight the need to carefully negotiate realistic contracts and the information 
asymmetries and knowledge gaps linked to this, as well as other issue such as difficulties encoun- 
tered cancelling contracts or preventing contractees from simply reneging on their contracted 
obligations (Jensen 2017). 
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Non-State and Society-Based Tools: Co-production and Certification 


Efforts at policy reform have been omnipresent in many developed and developing countries 
over the past several decades and have often featured efforts to reduce the number of state-based 
tools and shift their activities either towards hybrid instruments or, in some cases, away from 
state-based organizations altogether. Many of these efforts have featured waves of management 
reforms and administrative restructuring, including privatizations, deregulation and re-regulation 
and the like (Ramesh and Howlett 2006). “Anything but the government”, for example, was 
a popular sentiment in public policy reform for at least two decades (Christensen and Laegreid 
2008). These efforts have led to the articulation and promotion of several alternative modes of 
governing to more traditional “hierarchical” or state-led ones. 

Many of these techniques are “market based” and constitute efforts to replace government 
activity with private sector actions (Savas 1987). However, others are focused less on zero- 
sum notions of state-market relations but rather involve more complex ideas about involving 
“civil society” actors more directly in “collaborative” policymaking, administration and imple- 
mentation (Brudney 1987; Salamon 1989). These kinds of activities come in many forms, but 
two which have received a great deal of attention in recent years are “co-production” and 
“certification”. 


Certification 


Certification is a term used to capture the activities of many non-state actors involved in areas such 
as forestry, fisheries, organic foods and other similar areas in which quality control and enforce- 
ment of standards are accomplished less directly than is the case with traditional command- 
and-control-regulation (Cashore et al. 2004; Cutler et al. 1999; Gulbrandsen 2010). In these 
cases, for a variety of reasons from cost to ideology, “certification” of standards is undertaken by 
civil society organizations such as the Forestry Stewardship Council or the Marine Stewardship 
Council which lack the formal authority to compel business and industries to abide by regula- 
tory standards but which utilize (often negative) publicity, boycotts and other actions to encour- 
age compliance (Cashore 2002; Pérez-Ramirez 2012). 

These tools are often referred to as non-state market-based (NSMD) tools (Cashore 2002) 
since they do not rely on state authority for their power and legitimacy to regulate private sec- 
tor activity, but rather do so through market activities such as product labelling and producer 
certifications, affecting consumer behaviour and preferences for, for example, organic produce 
or sustainably harvested timber, fish or coffee, among others. 

Such schemes rely heavily on the reputation of the certifier for honesty, accuracy and pre- 
cision. In the case of certification, legitimacy and trust are key aspects of the certifying orga- 
nizations and predictors of the success of voluntary certification arrangements (Bernstein and 
Cashore 2007). Certification only functions effectively if trust exists between the public and 
certifiers and between the certifiers and certified companies and governments. Concerns about 
second-class regulation or corrupt standards can easily undermine years of work building up a 
certified brand. Similarly, competing or duelling certifiers can also undermine existing schemes 
and lead to their ineffectiveness (Zelli et al. 2017). If their reputation is damaged, as has hap- 
pened from time to time with products such as wine or olives in which additives were added 
or cheaper products substituted for expected ones, such schemes can collapse and require either 
substantial reform or government takeover, revealing their dependence on government, ulti- 
mately, to serve as the guarantor of quality. 
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Co-production 


Co-production is a shorthand term for a variety of governance arrangements which involve citi- 
zens in the production and delivery of public services. First set out by Elinor Ostrom (Parks 
et al. 1981) in her work on community policing in the 1960s, the term has been expanded and 
popularized by Victor Pestoff and Taco Brandsen in their many books and writings in the years 
since (Brandsen and Pestoff 2006; Pestoff et al. 2012). In the US, these ideas generated interest 
among public administration scholars in the 1970s and the 1980s and experienced a revival in 
the decades after the turn of the century (Pestoff et al. 2012). The idea has since been picked up 
and studied by some scholars around the world. 

Originally, co-production was narrowly defined as the “involvement of citizens, clients, con- 
sumers, volunteers and/or community organizations in producing public services as well as 
consuming or otherwise benefiting from them” (Alford 1998:128). Drawing on the experience 
of countries in Scandinavia in areas such as parent-teaching associations, Brandsen and Pestoff 
have highlighted the extent to which many governance arrangements, even those thought to 
be purely hierarchical, such as public schooling, in fact combine aspects of hierarchical or state- 
based governance with elements of civil society mobilization (Pestoff 2006). 

In early studies of activities such as parent-teacher interactions, this involvement in co-pro- 
duction activity was typically voluntary, meaning it existed as a positive externality reducing 
production and delivery costs of public services. This made it very attractive to governments 
seeking cost reductions in public service delivery, especially ones favourable to notions of “social 
enterprise” and enhanced community participation since as an end or good in itself (Parks et al. 
1981; Salamon 1981, 1987). 

Like all other collaborative tools, co-production also has a downside as a policy tool. In the 
case of co-production, it has long been recognized that expectations of free labour from co- 
producers may not materialize (Sorrentino 2015; Brudney and England 1983), and schemes 
to incentivize co-producers through payments are susceptible to all the usual harms of pub- 
lic expenditures, including corruption, clientelism and goals displacement, amongst others 
(Howlett 2017). 


Procedural Organizational Instruments 


Substantive tools, of course, are only half of the uses towards which government and non-gov- 
ernmental organizational resources can be put. The second use is procedural. This involves the 
use of the organizational resources such as personnel, staffing, institutionalization and internal 
procedures to alter or affect policy processes in order to better achieve general government aims 
or specific programme activities. 

It bears repeating that these tools do not involve direct or indirect goods and service delivery 
mechanisms, as do their substantive counterparts, but rather affect procedural activities, gener- 
ally efforts aimed at creating or restructuring policy community structure and/or behaviour 
through government leadership or “network management” efforts. 

According to Agranoff and McGuire (1999:21), this latter activity involves “network manag- 
ers” in “selecting appropriate actors and resources, shaping the operating context of the network 
and developing ways to cope with strategic and operational complexity”. The key dimensions 
or tasks involved in these kinds of network management activities include the identification of 
potentially compatible network actors, given the issue at hand; limiting potential conflicts that 
would hinder flexibility; recognizing legal requirements; balancing political objectives/conflicts 
with policy objectives; and assigning costs in implementation. 
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In any policy process, policy managers need to work with the structure and operation of 
any network which already exists in the area, recognize potential new actors, limit the role of 
ineffective actors, balance their time and resource commitments (money, technology, expertise 
etc.), maintain the focus of the network in achieving goals and build trust between actors/ 
reduce possible conflicts (Mandell 1994, 2000). In order to achieve these ends, various kinds of 
organizational network management tools can be used. Several of the most common of these 
are set out next. 


Public Sector Network Management Tools 


There are many different types of procedural tools linked with the use of specific government 
organizational resources which can affect various aspects of policy subsystem behaviour in policy 
processes. Interest in these tools has grown as many governments have moved in the direction of 
more overt network management in some sectors in recent years. 

In general, these agencies undertake tasks such as 


Vertical and horizontal agency coordination 

Overcome institutional blockages like federalism and divisions of power 
“Mainstreaming” 

Building commitments 

Building legitimacy/developing visions and agreement on alternatives 
Building coalitions 


pel Oy Ue N S 


Structuring NGO activity, e.g., lobbying activities 
(Mandell 2000, 185—210) 


A fairly commonly used procedural organizational tool is establishing new government agen- 
cies or reforming existing institutions in order to focus or refocus state and societal activities 
on specific problems or issue areas (Goetz 2007; Durant 2008). Setting up a new government 
ministry for technology or a new research council to promote advanced technologies like bio- 
technology, e-technologies or other high-technology sectors, for example, is a common action 
on the part of governments wanting to target a new area of activity for further development 
(Hood 2004; Lindquist 2006; van Thiel 2008). However, such actions are highly visible and, if 
repeated too often, quite costly. They are also quite intrusive and, as a result, are proposed and 
used only infrequently. Nevertheless, these tools are employed by all governments. They include 
the following typical sorts. 


Staff or Central (Executive) Agencies 


This is an old form of government organization, one in which a small, coordinating govern- 
ment agency, rather than one which directly delivers services to the public, is created to central- 
ize agency initiatives in some area. Such “staff or “central” agencies are generally created as a 
means to control other administrative agencies and are often linked very closely to the political 
executive (Bernier et al. 2005). In Westminster-style parliamentary systems, for example, older 
examples include privy council offices and treasury board secretariats while newer ones include 
presidential, premiers’ and prime ministers’ offices; ministries of state; communication units; 
intergovernmental secretariats; and various kinds of implementation units (Chenier 1985; Savoie 
1999; Lindquist 2006). They have seen much growth in recent years as political executives have 
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sought to re-establish control over far-flung administrative apparatuses (Campbell and Szablow- 
ski 1979; Rhodes and Weller 2001; Bevir et al. 2003; Bernier et al. 2005). 

Unlike line departments, these staff or central agencies are less or non-hierarchical, flatter 
organizations typically stafted by political appointees, although others also employ permanent 
officials as well. Key officials are chiefs of staff, principle secretaries and specialized positions 
such as a clerk of the privy council or cabinet secretary. These agencies play a major and 
increasing role in designing and coordinating policies and policymaking, ensuring account- 
ability to legislatures and controlling the budgets, activities and plans of line departments and 
ministries. Their small cost is a major design consideration, although this is often offset by their 
high visibility and high level of intrusiveness in the affairs of the government agencies they 
control or coordinate. 


Tribunals and Other Quasi-judicial Bodies 


These are created by statute and perform many administrative functions, hearing appeals con- 
cerning licencing (e.g., of pesticides), certification (of personnel or programmes) and permits 
(e.g., for disposal of effluents). Appointed by government, they usually represent, or purport to 
represent, some diversity of interests and expertise. 

Administrative hearings are conducted by tribunals in a quasi-judicial fashion, often in order 
to aid tribunals in their activities. These hearings are bound by rules of natural justice, and pro- 
cedures may also be dictated by statutory provisions. The decisions of tribunals are designed to 
be binding on the ministry in question but may be subject to various political, administrative and 
judicial appeals. Public hearings may be statutorily defined as a component of the administrative 
process. 

In the framework of administration, tribunals are directed toward securing compliance with 
administrative edicts and the achievement of identified standards of behaviour by both gov- 
ernmental and non-governmental actors. They may act as a mechanism with which to appeal 
administrative decisions, but in most cases, proceedings are held at the discretion of a decision- 
making authority, and public hearings are often “after the fact” public information sessions 
rather than being true consultative devices (Grima 1985; Stewart and Sinclair 2007). 


Analytical Units 


Some governments have also set up internal think tanks or research institutes in order to pro- 
vide policy advice to governments (Dobuzinskis et al. 2007; Marchildon 2007). Many gov- 
ernment departments and agencies also have established specialized policy units designed to 
generate studies and reports which can influence or help persuade both government officials and 
non-governmental actors of the merits of government plans. These agencies also often employ 
outside consultants to bring additional expertise and knowledge to policy formation, implemen- 
tation and evaluation (Schwartz 1997; Perl 2002; Speers 2007). The knowledge they generate 
is used to inform internal policy-making processes and also to garner support for government 
positions from outside groups (Whiteman 1985, 1995). 

New analytical units such as those policy shops created in many jurisdictions in the 1970s and 
1980s in order to promote formal policy analysis and what is now referred to as “knowledge- 
based” or “evidence-based” policymaking are good examples of procedural organizational tools 
(Prince 1979; Prince and Chenier 1980; Chenier 1985; Hollander and Prince 1993; Lindquist 
2006). 
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Establishing Government Reviews, Ad Hoc Task Forces, Commissions, 
Inquiries and Public Hearings 


A sixth common procedural organizational tool used by governments is the establishment of 
a government review. These range from formal, mandated, periodic reviews of legislation and 
government activity by congressional or parliamentary committees and internal administrative 
bodies to “ad hoc” processes such as task forces or inquiries designed to activate or mobilize net- 
work actors to support government initiatives (Gilmore and Krantz 1991; Bellehumeur 1997; 
Marchildon 2007; Sulitzeanu-Kenan 2007, 2010; Rowe and McAllister 2006). 

Ad hoc task forces and inquiries are typically temporary bodies, much shorter term and often 
more issue related than institutionalized advisory committees. Ad hoc commissions are also 
created as instruments to consult a variety of interests with regard to economic and other areas 
of planning activity. These range from presidential or royal commissions to those created at the 
departmental level. Presidential and royal commissions are the most formal and arm’s length, 
and, therefore, are the most difficult for governments to control and predict and, therefore are 
used less often (Clokie and Robinson 1969; Chapman 1973; Flitner 1986; Salter 1990). 

The subject matter of an ad hoc commission is typically urgent, of concern to more than 
one ministry and level of government, and the subject of some controversy (Resodihardjo 2006; 
Sulitzeanu-Kenan 2010). It is invoked at the discretion of government and is subject to political, 
economic and social pressures. Indeed, the very initiation of the commission is likely to be the 
product of pressure by public interest groups. But, as Chapman (1973:184) noted, “Commis- 
sions may also play a significant political role and are often used as a method for postponing to 
an indefinite future decision on questions which appear to be embarrassing but not urgent”. 
Employment of these instruments for this purpose can result in serious legitimation problems for 
governments utilizing these policy tools, however, given their high level of visibility (Heinrichs 
2005; Hendriks and Carson 2008; Stutz 2008; Marier 2009). 

Public participation through hearings is the most common type of public or network con- 
sultation in many sectors (Rowe and Frewer 2005). Hearings vary by degree of formalization 
and when they occur in a policy process. The most effective and influential are often flexible 
processes that are geared towards policy formulation, such as project reviews or environmental 
assessments, but the most common are rigid processes that take place in or after the imple- 
mentation stage of a process, such as a formal policy evaluation exercise (Dion 1973; Baetz and 
Tanguay 1998; Edelenbos and Klijn 2005). Public hearings are often mandated by legislation 
and most often occur after a decision has been taken — that is, purely as information and/or 
legitimation devices. Actual instances of open, truly empowered public hearing processes are 
very rare (Riedel 1972; Grima 1985; Torgerson 1986). 


Legislative and Executive Oversight Agencies 


This category of organizational tools also includes specialized agencies with very different 
policy-making functions, like arm’s-length independent auditor generals or access to informa- 
tion commissioners, which are units typically attached to legislatures, providing some over- 
sight or control over executive branch activities (Campbell-Smith 2008). Many principle-agent 
problems can also be overcome through administrative procedures mandating oversight agency 
reviews of government actions (McCubbins and Lupia 1994; McCubbins et al. 1987), espe- 
cially if these are linked to funding and budgetary issues (Hall 2008). These latter units are 
usually fairly small, inexpensive and highly visible, and there has been a proliferation of such 
units in recent years dealing with areas such as corruption, human rights and the promotion 
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of ethnic and gender equality (Malloy 2003). They often represent an effort to promote legal 
governance in sectors typically configured in other modes. 


Conclusion: Organizational Tools — The Forgotten Fundamental 
in Policy Design Studies 


In recent years, for a variety of reasons, a strong preference for shifts towards non-hierarchical 
forms of governance coupled with discontent with the results of market-based reforms in the 
1970s to 1990s led to increasing attention being paid to different kinds of more civil-society 
or networked forms of governance, often referred to by the shorthand term “collaborative 
governance” (Meuleman 2009). Many of the recent innovations in organizational forms, from 
special operating agencies to quangos, public-private partnerships and various kinds of hybrid 
organizations, have emerged largely in the effort to reduce the size of existing organizations and 
transform some sectoral activities from legal and corporatist to market modes of governance 
(Hardiman and Scott 2010). This is done in the name of improving the efficiency of service 
delivery or in order to try to reduce the resource burden large public service delivery agencies 
place on budgets and taxpayer loads (Verhoest et al. 2007; O’ Toole and Meier 2010). 

Often lost in the identification of alternative forms of collaboration followed by governments 
around the world in a variety of sectors, however, has been the observation that all collaborative 
modes are vulnerable to specific kinds of failures due to these inherent vulnerabilities. When 
governments reform or try to shift from one mode to the other modes, they need to understand 
not only the nature of the problem they are trying to address and the skills and resources they 
have at their disposal to address it, but also, especially, the innate features of each potential gov- 
ernances mode and the capabilities and competences each requires in order to operate at a high 
level of performance. 

If these critical capacity deficits are not taken into account, then any short-term gain enjoyed 
by deploying a hybrid collaborative governance technique is likely to be cancelled out later 
when the consequences of policy failures and poor institutional design due to its governance 
limitations become apparent (Hood 2010; Weaver 1986). 

Traditional organization-based implementation tools such as government agencies and pub- 
lic enterprises, on the other hand, are generally costly and high visibility because they rely on 
government personnel funded by appropriations from general revenue raised through taxes or 
royalties, although some are also funded from market revenue stemming from the sale of goods 
or services. But they are reliable and remain “the workhorses” of government. Despite their 
real or perceived cost, and in spite of many efforts to create or replace them with other forms 
of service and goods delivery, however, direct delivery of goods and services by public agencies 
remains what Christopher Leman (1989) has called “the forgotten fundamental” of policy tools 
and designs. 
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IMPLEMENTING REGULATORY 
POLICY 


Regulatory Tools and the Use of Authority 


Eric Windholz 


Regulatory implementation, simply put, is the carrying out of regulatory policy. However, 
regulatory implementation is far from simple. Traditional conceptions of regulatory implemen- 
tation that rely solely on the coercive authority of the state to command and control are out of 
favour. Today’s concept of regulation is far more expansive. It includes a wide range of regula- 
tory tools and instruments ranging from the compulsory to the voluntary, from the legal to the 
behavioural. It also recognises that the state is not the sole locus of regulation and that regulation 
is increasingly polycentric, networked and collaborative. This chapter explores the different 
types of regulatory tools that have evolved to implement public policy and the different and 
nuanced regulatory strategies and regimes through which they are implemented. 


Introduction 


This chapter is about implementing regulatory policy. But what is ‘regulatory policy’? Lowi, in 
his seminal 1972 article “Four Systems of Policy, Politics, and Choice’, defined it as policies that 
employ regulatory techniques to modify individual or collective behaviour to achieve a specific 
purpose (the other policy types being distributive, redistributive and constituent) (Lowi 1972). 

While this definition has withstood the test of time (see, e.g., Black 2002; Koop and Lodge 
2017), many of the ‘regulatory techniques’ of today would barely be recognisable to Lowi. What 
constitutes regulation has been ‘rethought’ over the past 50 years (Hodge 2021). Traditional 
state-centric, rule-based conceptions of regulation have given way to a more expansive, plural- 
istic and collaborative concept of regulation. 

With this rethinking have come new regulatory tools, strategies and regimes that extend 
beyond the state and the law. Government regulation has evolved from traditional ‘command 
and control’ models comprising prescriptive rules coercively enforced by the state to regulatory 
models variably described as ‘responsive’ and ‘smart’, employing tools designed to persuade, 
assist, incentivise and even nudge compliance with standards that are increasingly performance 
based and outcome orientated (Windholz 2018). Nor is government the sole locus of regulatory 
activity. Regulation today is increasingly polycentric, networked and collaborative (Black 2008; 
Klijn 2008; Talias 2021), shaped and co-produced through interactions between government 
and non-government actors, recognition that both industry and civil society regulate in addition 
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to government (Grabosky 1995, 2013; Scott 2001; Abbott et al. 2017). This chapter explores 
this evolution in regulatory theory and practice. The chapter first explains and examines the new 
regulatory tools, strategies and regimes to which the rethinking of regulation has given birth.! 
The chapter then asks what this says about the use of authority in the modern state. 


Regulatory Tools 


As our understanding of regulation has evolved and expanded, so, too, have the regulatory tools 
and mechanisms at the modern regulator’s disposal. Over time, different systems for classifying 
these tools and mechanisms have been developed. Many are instrumental in nature. Freiberg 
(2010), for example, identifies six sets of regulatory tools: legislation, economic, transactional, 
authorization, informational and structural. However, given that our conception of regulation 
focuses on behaviour modification, categorising regulatory tools by reference to the mecha- 
nisms through which they seek to trigger behavioural responses (rather than their form) is more 
instructive. 

Five key behaviour modification mechanisms can be distilled from the literature (see, e.g., 
Schneider and Ingram 1990; Daintith 1994; Vedung 1998; Morgan and Yeung 2007). They 
are coercion, incentives (and disincentives), persuasion, assistance and nudges. Few regulatory 
tools are one dimensional, however. Most tools can be thought of as a hydra, simultaneously 
triggering more than one behavioural response to effect the desired regulatory change. Never- 
theless, it is instructive to examine them individually to better understand their nature and key 
characteristics. 


Coercion 


Coercion is the behaviour change mechanism most commonly associated with regulation. Coer- 
cive mechanisms generally have three elements: (1) rules defining standards that must be met, 
(2) mechanisms to detect non-compliance with those rules and standards and (3) sanctions that 
attach to those non-compliances. Some rules are consensual and accepted voluntarily. Examples 
include contracts and industry codes of conduct. The sanctions that attach to breaches of these 
come in the form of contractual, social and market penalties. However, coercion is most com- 
monly associated with state-made and imposed rules. The sanctions that most commonly attach 
to non-compliance generally fall into one of two categories: criminal (e.g., imprisonment, fines, 
community service, probationary and supervisory orders and the like) and civil or administrative 
(e.g., fines and other monetary penalties; infringement; improvement and prohibition notices; 
and licence conditions, suspensions or revocations). 

Coercive mechanisms have a number of advantages. Compared to other behaviour change 
mechanisms, coercive mechanisms are more straightforward to formulate and serve a strong 
expressive role. They communicate that certain behaviour is considered unacceptable and, in 
the case of state-imposed rules, do so with the authority of the state. Coercive mechanisms also 
can be designed to operate with immediate effect, thereby enabling governments to position 
themselves as strong and decisive protectors of the public (Baldwin et al. 2012: 107). Coercive 
mechanisms also have disadvantages. They frequently are criticized for being blunt and over- 
inclusive and for imposing unnecessarily high compliance costs on those subject to them. 

Coercive mechanisms also can be expensive to administer and enforce. They require systems 
to detect and correct instances of non-compliance. Generally, these come in the form of inspec- 
torates, audits and other methods of policing the conduct of regulatees and administrative and 
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judicial mechanisms to correct and sanction non-compliance. These systems can become adver- 
sarial and legalistic, resulting in increasingly formal, expensive and lengthy processes. This, in 
turn, can undermine constructive regulator-regulatee relationships and the beneficial coopera- 
tion they can produce or, worse, breed defiance and resistance that can lead to non-compliance. 
The advantages and disadvantages of coercive mechanisms combine to suggest they are best 
applied to issues of higher risk, impact and/or significance with respect to which the community 
expects strong and decisive state action. For issues that do not meet these criteria, more flexible 
and facilitative mechanisms may be appropriate. It is to these mechanisms that we now turn. 


Incentives (and Disincentives) 


Incentives (and disincentives) assume individuals make rational benefit-cost decisions on whether 
to comply with regulatory standards. They then operate to influence those decisions by increas- 
ing the benefits and/or decreasing the costs of compliant behaviours (through incentives) or 
reducing the benefits and/or increasing the costs of non-compliant behaviours (through disin- 
centives). Incentives and disincentives can be direct and indirect. Examples of direct incentives 
and disincentives include tax credits, differential tax rates, rebates, bounties, grants, subsidies and 
levies that confer a direct financial benefit or detriment on the regulatee. Examples of indirect 
incentives include awards and certification systems that can be used by regulatees to differentiate 
themselves positively in the marketplace. Government contracts also can operate as an indirect 
incentive by making compliance with certain standards a prerequisite to doing business with 
government. 

Establishing incentives tends to be more complex and challenging than establishing a coer- 
cive regime. Incentives need to be carefully targeted at the right actors and at the right level to 
incentivise desired behaviours while avoiding gaming and rorts that can doom regulation to fail- 
ure. This careful targeting often finds its way into detailed, complex and sometimes prescriptive 
rules and standards which generally require sophisticated audit and compliance mechanisms to 
ensure non-compliance is detected, and manipulation and rorting of the system are prevented. 
The need for these mechanisms can result in incentive mechanisms coming to resemble coercive 
mechanisms. As Braithwaite (1982: 595) observes, the use of incentives ‘does not eliminate the 
need for government inspectors; it merely replaces inspections by technical experts with inspec- 
tions by experts in financial deception’. 

Once established, however, many incentive mechanisms are largely self-executing, the tailor- 
ing of the incentive to individual circumstances having been done at the design stage. This can 
make them cheaper to administer. They also allow for greater flexibility and innovation on the 
part of regulatees — incentivizing them to go beyond compliance with standards that might only 
be met under a more coercive and prescriptive regime. This assumes, of course, an economically 
rational regulatee. Compared to coercive mechanisms, incentives are less likely to operate on 
the irrational, irresponsible and ill informed who arguably are in the most need of having their 
behaviour modified. Nor are they likely to operate with the same immediate effect as coercive 
mechanisms. Understanding the nature of an incentive and how best to modify operations to 
take advantage of it can take time. Finally, incentives can lack the expressive clarity of coercive 
mechanisms, especially when directed at the producers of socially harmful activities such as pol- 
lution. Whereas coercive mechanisms unambiguously communicate the unacceptability of the 
activities they prohibit or restrict, incentives can appear counterintuitive by financially support- 
ing the very activity that is sought to be eliminated or reduced. 

Incentives operate to modify behaviour through the provision of tangible benefits and costs 
that appeal to regulatees’ hip pockets. But there are other motivational forces to be triggered. 
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Persuasion 


Regulation can seek to persuade persons and entities to change their behaviour by appealing to 
their interests, values, beliefs and preferences. Persuasive mechanisms seek to influence behav- 
iour without recourse to coercive or incentive-driven interventions. Rather, persuasion works 
by cajoling, exhorting and/or convincing the regulatee to act in a manner consistent with the 
regulatory objective (Yeung 2005: 369-371). This requires regulators to understand different 
regulatees’ different motivational drivers and behavioural triggers. Depending on the regulatee 
and its motivation to comply, this might involve an appeal to the regulatee’s self-interest (e.g., 
by providing new information that alters the way the regulatee perceives, weighs and balances 
the benefits and costs of alternative courses of action). In other cases, it might involve an appeal 
to the regulatee’s altruism or sense of community (e.g., by appealing to intangible values such 
as justice, equality, fairness or morality or to important societal concerns such as safety and 
security). And in others still, it may be an appeal to the regulatee’s instincts to comply with (or 
exceed) societal norms. Studies have shown that providing people with data that compares their 
performance with the norm encourages those people to change their behaviour to improve their 
performance (see, e.g., Schultz et al. 2007). 

Persuasion, targeted properly and executed well, can be effective to change behaviour or, at 
least, to motivate people to consider changing their behaviour. But on its own, it is unlikely 
to lead to widespread and sustained change. Persuasion works best when coupled with other 
behaviour change mechanisms, whether that be the reinforcing deterrent effect of coercive 
mechanisms or the benefits of an incentive. 

Coercion, incentives and persuasion all assume the regulatee needs to be imposed upon, 
incentivised or convinced in some way to behave in accordance with regulatory standards. In 
some instances, however, the regulatee wants to comply but does not have the knowledge, skills 
or resources to do so. In this situation, what they require is assistance. 


Assistance 


Assisting regulatees to understand and comply with regulatory standards is an integral part of 
modern regulatory regimes. Like other behaviour change mechanisms, assistance can come in 
many forms. The most common forms are information and advice, training and education and 
resources and infrastructure. 

Assistance mechanisms have an important advantage over other behaviour change mecha- 
nisms: they are practical and direct. Whereas coercive mechanisms tell regulatees what to do 
and incentives and persuasion encourage them to do it, assistance mechanisms help them do 
it. Of course, to be effective, assistance mechanisms need to be well targeted and effectively 
implemented. Information and advice and training and education need to be carefully prepared 
to ensure they are relevant and intelligible to those to whom they are directed. This, in turn, 
requires those communicating the information understanding their (different) audiences’ exist- 
ing levels of understanding, knowledge and needs. Information also needs to be provided to 
people cognisant that, for some, their ability to process and absorb it is limited. Providing people 
with too much information, or information that is too complex, can overwhelm, intimidate and 
cause people to disregard the information or, worse, to disengage from the regulatory process 
(Weil et al. 2006). 

Assistance mechanisms assume regulatees are motivated to comply, whether by coercion, 
incentives or persuasion. Assistance mechanisms also assume regulatees will comply if prop- 
erly informed, skilled and resourced. This is a rational assessment based on regulatees acting 
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rationally. But not all decisions are rational. Our rationality is bounded, and we act under various 
cognitive biases. This is where nudge techniques come in. 


Nudge 


What actually constitutes a ‘nudge’ is a question of some controversy. Although born out of the 
resurgent interest in behavioural economics as a basis of regulation, nudges are more than tradi- 
tional regulatory tools behaviourally informed (although arguably, all regulatory tools should be 
behaviourally informed). The most frequently cited definition of a ‘nudge’ comes from Thaler 
and Sunstein (2009: 6), whose book Nudge: Improving Decisions about Health, Wealth, and Hap- 
piness made nudges the regulatory technique du jour. Their definition provides that a nudge 


is any aspect of the choice architecture that alters people’s behavior in a predictable way 
without forbidding any options or significantly changing their economic incentives. 
To count as a mere nudge, the intervention must be cheap and easy to avoid. Nudges 
are not mandates. 


Three features are important in this definition. First, nudges focus on altering the “choice 
architecture’ — the context and environment in which decisions are made. Second, nudges alter 
people’ behaviour in predictable ways. They do this by dispensing with the neoclassical fiction 
that human beings are rational self-utility maximisers and replacing it with the empirical reality 
that our rationality is bounded by our habits, engrained heuristics and cognitive biases that result 
in systematic deviations from standards of rational decision-making (Simon 1945; Kahneman 
et al. 1982; Kahneman 2011). Important cognitive biases identified by behavioural economists 
include present or current moment bias (the tendency for people to have a stronger preference 
for more immediate payoffs relative to later payoffs), status quo bias (the tendency of people to 
like things to stay the same), inertia bias (the tendency of people when given a choice to act or 
to do nothing, to do nothing) and omission bias (the tendency to judge harmful actions as worse 
and more morally reprehensible than equally harmful omissions) (see also Baron 2007; Ariely 
2008). And third, nudges respect the autonomy of the individual by leaving them free to choose 
a different — ‘not right’ — path. Examples of successful architecture-changing nudges include 
increasing the number of people choosing to consume healthy food by placing it at eye level 
and in easy reach (e.g., at the cash register desk) (Kroese et al. 2016), increasing the proportion 
of people agreeing to donate their organs by redesigning organ donation forms from opt in to 
opt out (Johnson and Goldstein 2003) and eliciting more honest disclosures on official forms by 
positioning the declaration and signature box at the start of a form rather than at the end (Shu 
et al. 2012). 

Nudge techniques (and the field of behavioural economics on which they are based) have 
found favour with many governments in recent years. Behavioural insights (or economics) teams 
have been established in the White House Office of Information and Regulatory Affairs (to 
which Sunstein was appointed administrator), the United Kingdom’s Cabinet Office (to which 
Thaler acts in an advisory capacity), the Australian government’s Department of Prime Minister 
and Cabinet, and the European Commission’s Behavioural Studies for European Policies Pro- 
gram (Kosters and Van der Heijden 2015: 277). 

Nudge techniques promise governments effective outcomes with comparatively little cost 
and bureaucracy and without overt restrictions on individual liberty. Nudge techniques are not 
without their critics, however (see, e.g., Hausman and Welch 2010; Rebonato 2012; White 
2013). Many nudge techniques rely for their effectiveness on their covert nature. Why? Because 
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drawing people’s attention to the fact that the context in which they are making decisions is 
being altered to influence them to make a choice other than that which they might otherwise 
have made can lead to rebellious and counterproductive behaviour. Their covert nature, how- 
ever, raises issues of transparency, accountability and ultimately, legitimacy. There is a fine line 
between nudge and shove. 


Regulatory Strategies 


Listing regulatory tools tells us little about how they are (or should be) be used. Are they 
employed with a ‘heavy’ or ‘light’ hand, proactively or reactively, rigidly or flexibly? The answer 
to these questions resides in the regulatory strategy adopted by the regulator. A regulator’s choice 
of strategy is a powerful statement to regulatees and other actors about how they can expect the 
regulator to interact with them. 


Deterrent and Facilitative Approaches 


Early regulatory strategies — consistent with then-dominant state-centric ‘command and control’ 
paradigms — adopted deterrent approaches which, as the name suggests, focus on government 
efforts to deter non-compliance. Deterrence models posit that persons and entities are primarily 
motivated to comply by the fear of sanctions (legal, social and/or economic) that attach to non- 
compliance, with the deterrent effect being a function of the perceived severity, certainty and 
celerity of the sanction. The emphasis on perception here is important. As Freiberg (2010: 213) 
observes, modern deterrence is not an objective quality but depends on regulatees understand- 
ing the standards with which they must comply and the regulatory practices through which they 
will be enforced. This means that for maximum impact, coercive mechanisms should be coupled 
with persuasion, education and information that reinforces the importance of complying with 
the regulatory standards, the breadth and effectiveness of the mechanisms in place to detect non- 
compliance and the severity and celerity of the sanctions that attach to non-compliance. Indeed, 
today’s reality is that pure command-and-control regimes are unlikely to exist. Even modern 
police forces employ non-coercive methods to educate and encourage persons to comply with 
the law. 

This brings us to facilitative approaches. Facilitative approaches recognise that fear of sanc- 
tions is not the only factor motivating compliance. There is a rich body of research that empha- 
sises that compliance is a plural, dynamic and interactive process involving, in addition to classic 
deterrence theory, social, economic and normative dimensions. Social responsibility, reputation 
management, ethics, an internalized sense of duty and general law abidingness can all play an 
important role (Feldman 2011; Parker and Nielsen 2017). A facilitative implementation style, as 
its name suggests, focuses on persuading, encouraging and assisting regulatees to comply with 
their regulatory obligations. 

Early regulatory literature tended to present deterrent and facilitative approaches as distinct 
models from which regulators must choose. A one-dimensional one-act play. This changed 
when Ayres and Braithwaite (1992) put forward their model of ‘responsive regulation’. 


Responsive and Smart Approaches 


Responsive regulation provides for both deterrence and facilitative approaches in the one regu- 
latory regime. As Ayres and Braithwaite (1992: 25) explain: “The trick of successful regulation 
is to establish a synergy between punishment and persuasion’. Responsive regulation seeks to 
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achieve this synergy by matching the regulator’s response to the regulatee’s willingness and 
capability to comply, reflected in its compliance behaviour. To guide the matching of responses 
to motivations, Ayres and Braithwaite (1992: 35) developed the ‘enforcement pyramid’ with 
facilitative tools at the bottom, followed by progressively more coercive tools as one moves up 
the pyramid. According to Ayres and Braithwaite, regulation should start at the bottom of the 
pyramid and from the presumption that regulatees are virtuous and will respond to facilitative 
strategies such as persuasion and assistance. After this starting point, regulation becomes a dia- 
logic exercise. Regulatees who cooperate and behave as desired remain at the bottom of the 
pyramid. However, regulatees who do not cooperate and who fail to behave as desired are met 
with an escalation of enforcement activity. Escalation continues up the enforcement pyramid 
until compliance is achieved, at which point a consistent demonstration from that regulatee that 
it is now inclined to comply and behave as desired may be responded to by a de-escalation of 
regulatory activity down the pyramid and a less punitive approach. Ayres and Braithwaite refer 
to this as ‘tit-for-tat’ enforcement. 

But responsive regulation went further than just ‘tit-for-tat’ regulation of business by govern- 
ment. It also argued that governments could and should delegate regulatory tasks to business 
and provided a pyramid of enforcement strategies to guide the delegation process, with industry 
self-regulation at the bottom, government command-and-control regulation at the top and 
co-regulation in between (described as ‘enforced self-regulation’) (Ayres and Braithwaite 1992: 
39). However, Ayres and Braithwaite also were alert to the risks of capture and corruption that 
can attach to bilateral relationships that involve only regulator and regulatee. To guard against 
this, responsive regulation introduced the concepts of ‘tripartism’, in which the government 
regulator, those being regulated and persons representing those for whose benefit the regulatory 
regime exists cooperate in the development, administration and/or enforcement of the regula- 
tory regime, and ‘meta-regulation’, in which those third parties act as a countervailing force 
to the risk of regulatory capture and a check on the bilateral regulator-regulatee relationship. 
Examples include workplace health and safety regulatory regimes that include both employer 
and worker representatives and environmental regulatory regimes that include both industry and 
environmental public interest groups (Ayres and Braithwaite 1992: Ch 3). 

Responsive regulation represented a breakthrough in regulatory thinking and soon became 
‘one of the “canonical” texts that helped constitute the very field of which it is part’ (Parker 
2013: 2). One of the first significant attempts to build upon responsive regulation’s foundations 
was Gunningham and Grabosky’s Smart Regulation (1998). Smart regulation calls for an ‘imagi- 
native, flexible, and pluralistic approach’ to regulation (Gunningham and Grabosky 1998: 4). It 
builds upon responsive regulation in two important respects. First, smart regulation conceives of 
regulation beyond the state, recognising the role played by both professional bodies and industry 
associations as self-regulators and commercial and non-commercial third parties as quasi-or sur- 
rogate regulators. Second, smart regulation calls on all three sets of actors to regulate responsively 
and reconceives Ayres and Braithwaite’s two-dimensional enforcement pyramid into a three- 
sided pyramid, with escalation possible up each ‘face’ of the pyramid — the government face, 
the self-regulatory face and the third-party quasi-regulatory face — with each using different 
but complimentary regulatory tools. In so doing, smart regulation extols the virtues of using a 
mix of instruments over single-instrument approaches and argues for the use of complementary 
and sequenced combinations of regulatory instruments tailored to the causes and context of the 
specific issue. 

The logics underpinning responsive and smart regulation are difficult to refute. Complemen- 
tary combinations of tools should be more effective than single tools; involving the regulatee 
and third-party actors should be more effective than government acting alone; and tailoring 


322 


Implementing Regulatory Policy 


those tools and actors to the causes and context of each issue should be more effective than a 
one-size-fits-all approach. They also promise flexibility in enforcement approach, targeted use 
of limited resources and, when used transparently, a degree of procedural fairness and equity in 
application. However, the strength of their logic is matched by their difficulty to implement. 
Responsive and smart regulation ask a lot of regulators: that they have the time, capability and 
skills to interpret different regulatees’ compliance behaviour and to tailor complementary com- 
binations of interventions accordingly. Moreover, the ongoing dialectic relationship between 
regulator and regulatee on which they are based may not be appropriate in cases of immediate 
and catastrophic risk that demand immediate and decisive action (e.g., terrorism), often does not 
exist (e.g., because regulatees are many and dispersed), or can be compromised by enforcement 
action that makes de-escalation down the pyramid problematic (Windholz 2018: 233-236). And 
while the presence of non-state actors qua regulators can make implementation more effective 
and efficient, many moving parts must align to achieve their desired results (van de Heijden and 
Hodge 2021: 748). Nor are the risks of capture and corruption entirely eliminated, and issues of 
transparency and accountability are introduced. 

Notwithstanding these limitations, both responsive and smart regulation have been widely 
adopted by regulators around the world (Ivec and Braithwaite 2015; Gunningham and Sinclair 
2017: 143-146). They have been followed by other strategic approaches that seek to expand 
on the virtues of responsiveness, tripartism and the use of smart (complementary, tailored and 
sequential) mixes of tools and actors. Examples include really responsive regulation that argues 
regulators should be responsive to (in addition to regulatees’ willingness and capability to com- 
ply) the broader institutional environment, the logics of the tools at its disposal, its own perfor- 
mance and changes in the regulatory space in which it operates (Baldwin and Black 2008); really 
responsive risk-based regulation that combines really responsive regulation’s broader framework 
with the concept of risk-based regulation (Black and Baldwin 2010); regulatory arrangement 
approaches that seek to structure smart regulation’s regulatory options by reference to its insti- 
tutional environment (van Gossum et al. 2010); and behavioural responsive regulation that seeks 
to synthesize responsive regulation with behavioural economics (Barak-Cohen and Kariv-Teit- 
elbaum 2021). 


Regulatory Regimes 


Two concepts are central to responsive and smart regulation and the various strategies that have 
built upon them. First is the need for regulators to understand the motivations and capabilities 
of those whom they regulate and to choose a synergistic combinations of tools and strategies 
tailored to the context and needs of the particular situation. Second is to enlist and co-opt into 
the regulatory endeavour other actors who possess relevant resources. These resources could 
include information and knowledge, people and money, organisational capacities and networks 
and institutional credibility and political influence. Conceiving of regulation involving these 
other actors and their resources encourages us to think in terms of regulatory regimes. A regula- 
tory regime refers to the network of actors involved in regulating an issue; the regulatory tools 
employed by them to modify the behaviour of the target audience; and the norms, principles, 
rules and decision-making processes according to and through which those actors and tools are 
coordinated (Windholz 2018: 74). 

Over time, the nature, form and shape of regulatory regimes have evolved in conjunction 
with our evolving concept of regulation and with the complexity of issues with which societies 
must deal. Numerous labels have been coined and theories developed to describe and explain 
these regimes, with regimes today variably described as co-regulatory, polycentric, networked, 
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nodal and collaborative. These regimes are discussed next, starting with the traditional regime 
paradigms of government and self- and co-regulation. But before doing so, a word of caution for 
the uninitiated. Regime design is an area fraught with numerous taxonomies and differences in 
terminology. Some differences are subtle; many are not necessarily meaningful. The same term 
can be used to refer to different regime models and different terms to refer to the same regime 
model. It is an area in which it is essential to move beyond labels to understand the essence of 
the regime to which the label is being attached. 


Government, Self- and Co-regulation 


Government regulation is where government is solely responsible for making the rules; facili- 
tating, monitoring and enforcing compliance with the rules; and sanctioning non-compliance 
with them. Self-regulation, on the other hand, is when the regulatees themselves are responsible 
for developing and implementing the regulatory regime. Self-regulation can exist as an enter- 
prise level, at which each entity regulates its own affairs, or at a collective level, at which enter- 
prises voluntarily submit to regulation by a trade or industry association, such as in the form of 
a code of conduct (Sethi 2011). 

Government regulation often is criticized for being expensive to administer and costly to 
enforce; for imposing high and unnecessary compliance burdens on regulatees; for being com- 
plex, inflexible, stifling and unnecessarily broad; and for being prone to capture. Self-regulation 
is said to address many of these problems. However, it, too, has its disadvantages. It is criticised 
for operating in the private interests of those who develop and implement it, that it is misused to 
erect barriers to entry and facilitate anti-competitive conduct, that it is frequently a paper tiger; 
and that it lacks transparency and accountability (Bartle and Vass 2005; Priest 1998). 

Co-regulation — in which government and regulatees cooperate in the development and 
implementation of the regulatory regime — promises the best of both government and self- 
regulation. By empowering regulatees and co-opting their expertise and infrastructure, co- 
regulation promises regulation that is better targeted, more flexible, less burdensome and more 
effective than government regulation, with the protection that government involvement pro- 
vides against the system being captured, compromised or manipulated by regulatee self-interest. 

The variety of co-regulatory regimes are too many to catalogue. For present purposes, a 
few examples will suffice, starting with what Ayres and Braithwaite referred to as enforced 
self-regulation but which equally could be called enforced co-regulation. This is when govern- 
ment establishes the legislative framework and sets outcome-orientated standards, the regula- 
tee develops and puts in place systems to comply with those standards and the regulator then 
oversees and audits those systems to ensure they are appropriate and functioning effectively 
and sanctions non-compliance when they are not. Examples of enforced co-regulation are the 
safety case management systems increasingly being required of major hazard facilities and the 
food industry’s hazard analysis and critical control points (HACCP) system (Mortimore and 
Wallace 2015; Gunningham 2007). In other co-regulatory regimes, government establishes the 
legislative framework but delegates the standard-setting function to the regulatees. Regulation 
of professionals such as medical practitioners, lawyers and architects are examples of this type of 
model, in which governments appoint a professional body to be the standard-making institu- 
tion and makes compliance with its standards a prerequisite for market entry. Bartle and Vass 
(2005: 63) refer to this as devolved co-regulation. Closely associated to devolved co-regulation 
is delegated co-regulation. This is when government delegates standard setting and enforcement 
to a private body but retains an audit and appellate role to ensure the regime operates effectively 
and in the public interest. Stock market regulation is an example of delegated co-regulation. 
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Most stock market regulation is undertaken by privately run stock markets or exchanges that 
govern their members contractually but do so under the supervision of a government regulator 
with powers (that vary from jurisdiction to jurisdiction) to approve (and disapprove) exchange 
rules, to undertake inquiries into their operations and to hear appeals from their decisions (Priest 
1998: 259-261). 

Co-regulatory regimes have grown in number and prominence, partly in response to calls 
for more flexible and less prescriptive modes of regulation and partly in response to an increase 
in issues that are too complex and complicated for governments to regulate on their own. 
This complexity is also leading governments to think beyond traditional regulatory paradigms — 
which brings us to the next set of regulatory regime models. 


Polycentric, Networked and Other Collaborative Regulatory Regimes 


Polycentric, networked and collaborative modes of regulation look beyond government and 
those being regulated to include other actors in the regulatory domain. The simplest collab- 
orative model is tripartism which is when, it will be recalled, the government regulator, those 
being regulated and persons representing those for whose benefit the regulatory regime exists 
cooperate in the development, administration and/or enforcement of the regulatory regime. 
From simple tripartite models have been built polycentric and networked regimes of vari- 
ous complexities, comprising multiple actors performing a variety of regulatory roles. In these 
polycentric and networked regimes, regulation takes place through different actors (centres or 
nodes) that coordinate (or at least communicate) their activities through networks. In some 
cases, governments outsource specific regulatory functions to non-state actors (whom, it will be 
recalled, Gunningham and Grabosky described as surrogate or quasi-regulators). Examples of 
this are third-party certification schemes, such as private sector mechanics testing and certifying 
vehicle roadworthiness and private building inspectors verifying and certifying compliance with 
government-mandated building and construction standards (McAllister 2012; Grabosky 2013). 
In other cases, the role is less formal, such as when governments engage intermediaries (persons 
with strategic influence over the regulatee) to influence (incentivise, assist, persuade or nudge) 
the regulatee to modify its behaviour in accordance with the regulatory objective (Abbott et al. 
2017). An example of this is the use of medical and health professionals to communicate public 
health messages (Windholz 2018: 167). And in other cases still, regulatory responsibilities are 
assumed by non-government actors in the absence of government action, to which governments 
are later enlisted or co-opted. Examples of this are schemes for the verification and certifica- 
tion of environmental standards for forestry products and organic foods and labour standards for 
apparel (Bartley 2003; Grabosky 2013). 

The result is that regulation today is shaped and co-produced through interactions between 
government and non-government actors, and it is more likely to resemble a web than a linear 
two-way interaction between regulator and regulatee (Grabosky 1995, 2013; Scott 2001). An 
excellent example of such a web is Australia’s tobacco control regulatory regime. The primary 
regulatory targets — the regulatees — are smokers and persons contemplating smoking. The goal 
is to modify their behaviour so that they choose not to smoke. The regulatory regime comprises 
government actors employing regulatory tools directly targeting these persons such as coercive 
restrictions on where they can smoke, (dis)incentives in the form of price-increasing taxes and 
persuasion and assistance in the form of public health information and smoking cessation ser- 
vices. But there also are a vast array of non-government actors engaged in the regulatory endeav- 
our. Some of these other actors are incentivised to participate by government assistance in the 
form of information and funding (e.g., public health groups and medical professionals); some 
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are persuaded by public health information to act as agents of influence (e.g., role models, fam- 
ily and friends); others are coerced and compelled by government restrictions and requirements 
(e.g., tobacco manufacturers, retailers and hospitality venues); and others still act independently 
of government motivated by their own self-interest and the opportunities created by the regula- 
tory environment (e.g., insurance companies and plaintiff lawyers). Coordination of the regime 
actors and tools is achieved through the National Tobacco Strategy (network) developed by 
federal, state and territory governments in partnership with supportive non-state actors (Inter- 
governmental Committee on Drugs 2012). This dynamic is illustrated in Figure 25.1 (Windholz 
2018: 71-75, 167; author’s original). A picture paints a thousand words. The ‘web of influence’ 
is clearly visible. 

The advantages of collaborative, polycentric regulatory regimes are many. First, they can be 
more effective and efficient than more narrowly conceived regimes. More effective because they 
enable greater resources, knowledge and skills to be applied to achieving the regulatory objective 
and more efficient because those resources are not paid for by the state, freeing up state resources 
to be applied to those aspects of the regulatory endeavour that only government can (or should) 
perform (Gunningham and Grabosky 1998: 408—409). Including multiple actors also guards 
against the regulatory endeavour being captured by or unduly influenced by any one interest 
group. Their inclusion enables them to act as fire alarms should they detect regulatory drift, 
inefficiency, incompetence or self-serving behaviours on the part of the government regulator 
(McCubbins et al. 1987). Thought of in this way, the participation of non-state actors is a form 
of what has come to be known as meta-regulation — the external oversight of regulatory deci- 
sion-making. ‘Meta-regulation’ is another term to which different meanings are ascribed. Some 
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confine it to government oversight of government regulation. Others (including this author) use 
it more broadly to refer to any oversight of regulatory decision-making — be it by government, 
business or civil society (see generally Grabosky 2017). 

Including third parties as surrogate or quasi-regulators also can enhance regime legitimacy, 
especially if those third parties are perceived to be more credible and acceptable than govern- 
ment. In tobacco control regulatory regimes, for example, doctors may be perceived as more 
acceptable and credible anti-smoking messengers than governments addicted to the taxes smok- 
ers pay. 

Of course, these advantages come at a cost and with risk. Most obvious are the risks of 
capture and corruption: that these actors will use their involvement in the regime to advance 
their private interests and not the public interest. (Although, as we have observed, the presence 
of multiple actors guards against this risk.) These additional relationships also can make it more 
complex for regulators, those involved and the community to obtain a clear understanding of 
the performance of the regulatory regime and of the contribution of each actor to that perfor- 
mance. This can undermine accountability and transparency — what Thompson (1980) refers to 
as the ‘problem of many hands’. There also are administrative costs and risks. The more actors 
co-opted into the regulatory regime, the greater the complexity and the greater the costs of 
building and maintaining systems to coordinate their contributions in a manner that is consistent 
and synergistic. And with multiple actors comes risks of duplication, overlap and inconsistency 
if those systems do not function well. Polycentric regimes work best when those co-opted into 
the regime have convergent interests in achieving the regulatory objective. Absent convergent 
interests, third parties can become irritants and counterproductive. 


The Use of Authority 


Our examination of regulatory implementation has revealed that it has evolved from rules pro- 
mulgated and enforced by a sovereign state to a mix of innovative tools shaped and co-produced 
through interactions between government and non-government actors. In this section, we distil 
some observations about the implications of this evolution for the use of authority in the mod- 
ern state. 

First, the polycentric and networked regulatory regimes to which this evolution has given 
rise reflect that much regulation today is the product of government and non-government actors 
working collaboratively and horizontally, rather than the state operating hierarchically and verti- 
cally. This challenges the traditional assumption that the state is the primary locus for articulating 
community goals and values (Morgan and Yeung 2007). It also is recognized that state authority 
is not the only source of power within a regulated domain and that authority and power are 
frequently dispersed between a number of actors, public and private (Scott 2001: 330). Black 
(2008) goes further and suggests that governments compete with non-state actors for regulatory 
authority, legitimacy and influence. This theme is inherent in the literature on regulatory space 
or the ‘public policy arena’ (Hancher and Moran 1989). Parker (2000, 532), for example, refers 
to regulatory space as “a metaphor that emphasizes that the state must always compete with 
other regulatory actors for social control in any particular circumstances’. Morgan and Yeung 
(2007: 76) similarly refer to authority and power within a regulatory space being exercised both 
horizontally and vertically and by both state and non-state actors, thereby creating ‘a reflexive 
process of influence and change within [the] regulatory space’. Thus, in the tobacco control 
regulatory space, actors delivering the regulatory regime (Figure 25.1) also are seeking to modify 
the behaviour of (that is, regulate) government and each other to obtain policies favourable to 
their interests. 
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The sharing of regulatory power and influence amongst sometimes competing actors can 
have important consequences. A growing body of research has established that a regulator’s 
legitimacy (credibility and acceptability) is important to its ability to discharge its regulatory 
responsibilities and that regulatory compliance can depend significantly on people’s perception 
of the legitimacy of the regulatory regime and the regulators within it (Tyler 1990; Braithwaite, 
Murphy and Reinhart 2007; Murphy et al. 2009; Majone 1996: 11) goes so far as to suggest that 
legitimacy has replaced coercive power as the essential resource of policymakers. Yet as Levi- 
Faur et al. (2021: 6) observe, the legitimacy of regimes ‘is not necessarily rooted in state author- 
ity’. Legitimacy is multidimensional and also has normative, pragmatic and cognitive dimensions 
Suchman 1995; Black 2008; Windholz 2016). Legitimacy can be enhanced by co-opting third 
parties with credibility and institutional trust that governments may lack, but equally can be 


undermined by unhealthy competition and poor coordination among regime actors. 

Another important observation is that the authority inherent in a regulatory regime is imple- 
mented by individuals. When regulatory strategies refer to ‘the regulator’, they are not referring 
solely to the organisational entity. They also are referring to the persons who make the decisions 
required by that strategy. We have seen that implementing regulatory policy is not a matter of 
robotically applying automated or standardized responses. Regime activities are the product of 
the exercise by individuals of judgment and discretion. This discretion can be broad. Scott, for 
example, observes that many of the facilitative tools employed by regulators are not legislatively 
bestowed — they are the product of the creativity of policy and regulatory entrepreneurs (Scott 
2017: 271). This makes having processes, procedures and reward systems in place that variably 
coerce, incentivise, persuade, assist and even nudge regulatory staff to implement consistently 
with the regime’s strategy and objectives critically important. Regulation within government 
also needs to be responsive, smart and meta-regulatory. 

Evident in our discussion of the evolution of regulation is that the role of the state itself has 
transformed. Gunningham (2017: 139) refers to the role of government transforming to that of 
‘catalyst or facilitator’, creating the necessary preconditions for other actors to assume a regula- 
tory role. Others liken it to the conductor of a regulatory orchestra (Abbott and Snidal 2009). 
And while, in the public policy context, the state variably steers and directs such that regulatory 
policy remains under its auspices, it is not always under its control. In a world of polycentric 
and networked regulatory regimes in which all actors regulate responsively, government can find 
itself regulator in one context (and with respect to one set of actors) and regulatee in another 
(and with respect to another set of actors). 


Conclusion 


Regulatory implementation, simply put, is the carrying out of regulatory policy. However, as 
we have seen, regulatory implementation is far from simple. This chapter began by observing 
that over the past few decades, regulation has been rethought — that the traditional state-centric, 
law-based command-and-control conception of regulation has been replaced with one that 
is focused on ‘behaviour modification’ and in respect of which the state is not the sole locus 
of regulatory activity. This modern conception of regulation has taken hold among regula- 
tory scholars and practitioners, with profound implications. We have seen that it opened up a 
world of possibilities with respect to tool, strategy and regime design. The regulatory ‘toolbox’ 
is indeed rich — testament to the creativity, ingenuity and entrepreneurship of policy designers 
and regulatory practitioners. Of Lowi’s four systems of policy referred to in the introduction 
(regulatory, distributive, redistributive and constituent), regulation has grown in prominence and 
importance. For Braithwaite, Coglianese, and Levi-Faur (2007), regulation is the expanding part 
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of governance; for Hodge (2021: 727), it has become the policy preference of governments the 
world over. This makes better understanding of regulatory implementation critically important. 


Note 
1 Sections of this chapter draw from Windholz (2018). 
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TREASURE TOOLS 
Using Public Funds to Achieve Policy Objectives 


Bronwyn Mcllroy- Young, Daniel Henstra and Jason Thistlethwaite 


The public treasury can be used in many ways to implement public policy. In addition to direct 
spending on public goods and services, governments can use public funds to induce target groups 
to adopt policy-preferred behaviors or to impose costs on target groups to discourage undesirable 
behaviors. This chapter identifies different types of treasure tools and illustrates how they are used 
by governments to achieve policy objectives, drawing examples from different fields. 


Introduction 


Governments have access to a diverse array of policy instruments — tools they can use to imple- 
ment their policy objectives. These tools typically involve deploying authority and resources in 
different ways to influence the behavior of target groups, including individuals, organizations, 
and other governments. The public treasury — the funds collected through taxes, fees, and other 
revenue streams that are at the command of the political executive — is one of the key governing 
resources that can be harnessed for policy implementation. Treasure-based (i.e., economic) tools 
generally involve the use of public funds to produce and maintain public goods and services, to 
induce targets to adopt policy-preferred behaviors, and to impose costs on targets to discourage 
undesirable behavior (Howlett 2011, 101). 

This chapter focuses on economic instruments that governments use to implement public 
policy. It begins by discussing the general nature of economic policy tools and their inherent 
assumptions about the behavioral motivations of target groups. It then presents six specific 
types of economic policy tools, explains their underlying objectives, and provides examples to 
illustrate their use in practice. Each of the tools is also evaluated against criteria such as resource 
efficiency, coerciveness, and political risks. The third section discusses the use of economic 
policy tools in combination with other instruments to illuminate their potential in enhancing 
instrument mixes. The final section discusses the barriers to and limitations of economic policy 
tools, including some assumptions made about behavioral mechanisms and inequity. 


Treasure as a Policy Implementation Tool 


Achieving most public policy objectives requires enabling, convincing, or compelling people 
to do things they otherwise would not do (Schneider and Ingram 1990). Governments have 
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many tools to influence the behavior of target groups, and scholars have typically divided them 
into categories, such as information-based (e.g., public education, exhortation), authoritative 
(e.g., law, regulation), economic (e.g., tax, subsidy), and organizational instruments (e.g., coor- 
dination, facilitation) (Hood 1983; Howlett 2011). These tools are technically substitutable — 
instruments from any category could be employed to achieve the same policy objective — but 
they differ from one another in several different ways, including their underlying behavioral 
assumptions, how precisely they can target specific behaviors, the resources required to deploy 
them, and the political risks associated with their use (Landry and Varone 2005; Schneider and 
Ingram 1990). 

Economic policy tools are rooted in the state’s legitimate authority to tax citizens and spend 
the collective treasury in the public interest. This instrument category includes direct govern- 
ment spending on assets, programs, and services but also captures a range of discrete tools that 
aim to influence the behavior of target groups by creating financial incentives and disincentives 
(Henstra 2016). The latter set of economic tools, which are the primary focus of this chapter, 
are underpinned by a basic logic that targets will adjust their behavior to achieve financial gains 
or avoid financial losses. The use of these tools assumes that because targets are motivated by 
economic interests, their behavior can be influenced by attaching positive financial payoffs to 
socially desirable behaviors and imposing financial costs on undesirable behaviors (Olejniczak, 
Sliwowski, and Leeuw 2020). 

Grounded in this assumption, governments often employ grants, loans, and subsidies to entice 
target groups to engage in policy-preferred behaviors, the cost of which might be beyond their 
willingness or capacity to pay. Similarly, policy targets who align their actions with the govern- 
ment’s stated policy objectives are often offered reduced or waived financial obligations, such as 
fees and taxes, in exchange for their compliance. Conversely, achieving policy objectives often 
involves constraining behaviors deemed contrary to the public interest, so governments impose 
fees, tariffs, and taxes to attach costs to these behaviors, in hopes of altering the decision calculus 
of actors who might engage in them. 

Economic policy tools are favored by governments because of their targeting precision. 
Grants and subsidies, for instance, are typically deployed at an individual level, include report- 
ing requirements, and are often paid out only after the target has made an initial expenditure, 
so the costs of deployment are relatively easy to track, and the impacts can be measured fairly 
accurately. Similarly, since they are used to discourage known undesirable behaviors, taxes and 
fees can be imposed selectively to capture only those actors who engage in these behaviors. 

Economic tools therefore offer policymakers a flexible arsenal of instruments that can be 
finely tuned and calibrated to achieve policy objectives. These tools can also be deployed with 
one another or alongside other categories of policy tools to meet the needs of different stake- 
holders simultaneously and provide more robust solutions to policy problems. However, since 
mounting any of these economic tools imposes costs on the public treasury, their use is con- 
strained by governments’ fiscal capacity and ideological resistance to public spending. 


Treasure Tools 


This section analyzes a selection of treasure instruments that governments commonly use. Spe- 
cifically, it discusses direct program spending, grants and subsidies, loans, government insurance, 
fees and corrective taxes, and tax expenditures. For each tool, we evaluate its level of coercive- 
ness and resource efficiency, as well as the political risks and support involved in its implemen- 
tation. Also considered for each instrument are the specific mechanisms and calibrations that 
governments can use to focus their targeting or overcome administrative constraints. 
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Direct Program Spending 


Direct program spending involves the use of public funds to produce and maintain public goods. 
By designing and implementing programs, governments rely on their own capacity to accom- 
plish policy objectives, rather than acting through intermediaries to influence the behavior of 
others (Hood and Margetts 2007, 6). Public programs can take many forms. To support adapta- 
tion to a changing climate, for example, governments have implemented large spending pro- 
grams to replace and rehabilitate infrastructure and to improve heating and cooling in buildings 
(Henstra 2016). 

General program spending is fundamental to governance as a direct mechanism to implement 
policy objectives and to set the stage for other activities in a policy field. When well designed, 
executed, and monitored, government programs are generally effective, in that they can pro- 
duce more reliable outcomes than other types of treasure instruments that attempt to influence 
the behavior of external actors by providing incentives. However, government programs are 
resource intensive because they rely on public servants for implementation, and they carry a 
significant political risk because of their high visibility and cost to the public treasury. 


Grants and Subsidies 


Grants and subsidies are payments made from governments to individuals, companies, or other 
governments to achieve a public policy objective. Grants are direct payments that cover some or 
all of the costs of undertaking an activity, and they are among the most commonly used policy 
tools. They may be conditional, meaning that the funds are awarded on a competitive basis and 
can only be used for specific purposes, or unconditional, meaning that the funds can be used 
for any purpose desired by the grantee. Grants are intended to be “stimulative and temporary”, 
in that they enable or encourage an activity in hopes that the recipient of the payments will 
continue to provide it after the gifted funds are cut off (Beam and Conlan 2002, 341). They are 
a popular vehicle for national governments to implement their policy objectives by influencing 
the decisions and actions of other levels of government without overstepping their legal author- 
ity. Grants are also a mechanism for bridging the funding gap by providing fiscal support to 
lower levels of government that lack the capacity to meet their needs through taxation. 

Subsidies are payments to individuals and organizations that are intended to reduce the cost 
of policy-preferred behaviors or activities in hopes of increasing uptake among target groups. 
Unlike grants, subsidies are always conditional, in that they require performance of specified 
requirements and cover only a portion of the cost. Subsidies are often explicit, such as a city 
program to subsidize the installation of basement flood prevention devices, but can also be 
covert, such as a crop insurance program in which the premiums are set lower than the expected 
claims, effectively delivering a hidden subsidy to farmers. The latter exemplifies how grants and 
subsidies can serve redistributive goals. 

Both grants and subsidies can also be used to correct market failures by reducing the costs 
of activities that are considered socially beneficial to governments but are not sufficiently lucra- 
tive for firms (Steenblik 2018). For example, the PeakSmart program in Queensland, Australia, 
subsidizes the purchase of air conditioners that are designed to reduce electricity network stress 
during periods of peak demand, such as a heat wave (Australia 2021). While the societal benefits 
of increased network efficiency are significant, the subsidy is needed to increase the marketabil- 
ity of PeakSmart units because their benefit to individual consumers is indirect. 

Grants and subsidies are non-coercive because they provide economic incentives but rely 
wholly on voluntary uptake. These instruments are moderately resource efficient because they 
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are designed to achieve policy objectives by relying on individuals, organizations, or other gov- 
ernments to carry out actions (Beam and Conlan 2002). Grants tend to attract strong political 
support from vertical coalitions that include grant recipients, officials from the administering 
governments, and other stakeholders who believe in the need for ongoing support for their 
specific program. Similarly, subsidies are more politically palatable than unconditional transfers 
because they put some of the financial liability on policy targets (Steenblik 2018). 

Despite these advantages, grants can be quite inefficient when they require administration 
by multiple levels of government (Beam and Conlan 2002). Canada’s Municipalities for Cli- 
mate Innovation Program, for instance, provides funding to hire climate change adaptation 
and mitigation staff. It is funded by the government of Canada, delivered by the Federation of 
Canadian Municipalities, and implemented by municipal governments. A multi-tier implemen- 
tation structure of this sort is vulnerable to inefficiency and redundancy arising from numerous 
smaller programs with overlapping functions. Subsidies present additional efficiency concerns 
specifically related to economic distortion, which occurs when subsidies warp natural market 
incentives, resulting in a diversion of economic resources from more productive to less produc- 
tive uses (Steenblik 2018). 


Loans 


Loans are an economic instrument through which governments extend credit to fund bor- 
rowers in performing policy-preferred activities (Howard 2002). Government loans provide 
liquidity and direct access to capital for activities that are considered socially or economically 
valuable, such as the growth of small businesses or obtaining an undergraduate degree. They can 
be extended directly by the government or issued through a private bank with a government 
backstop. The latter, known as a guaranteed loan, involves a contractual agreement with the 
private lender whereby the government pledges to repay the loan to the private lender if the 
borrower defaults. 

The rationales for and effectiveness of loan programs are variable. Loans can be used to cor- 
rect capital market imperfections or to pioneer new credit markets (Howard 2002). For example, 
many European countries offer loans as an incentive to scale up energy efficiency measures that 
have high upfront costs (Economidou et al. 2019). The private loan market is underdeveloped 
in this area, and so governments intervene to offer public debt financing options. 

Loans have a low level of coerciveness in that signing on is voluntary. Once a loan has 
been extended, however, highly coercive actions may be taken to police delinquency or default 
(Howard 2002). The resource intensity of a loan program depends on whether it is implemented 
directly — and therefore requires government staff to administer — or through a private firm. 
Keeping a loan program solvent, rather than an open-ended, money-losing enterprise, is no 
small task. Effective loan programs require sophisticated information management systems and 
highly trained staff. Further, because of the long time horizons for many prepayment schemes, 
it may take decades to realize that lending was imprudent, and public funds have been distrib- 
uted injudiciously. As a policy instrument, loans attract general political support because the 
government is perceived to be acting like a business, though private lenders generally prefer 
loan guarantees over direct lending because they can introduce new customers to their services. 
In either case, constituencies may be wary of new loan programs because of their potential to 
disrupt existing markets. These same constituencies, once adjusted to public lending in a new 
market, can also be a source of inertia for ending or modifying loan programs. 

There are ways in which the administration and targeting of loan programs may be calibrated 
to efficiently meet policy objectives. Policymakers must decide the terms of the loan, including 
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who is eligible, whether to provide a form of subsidy (e.g., through a low interest rate), the size 
of the loan, the time for repayment, and the conditions of default. Effective government loan 
programs walk the line between “doing well” by targeting people who will not default and 
“doing good” by targeting people who will not be offered the credit they need from a private 
bank (Howard 2002, 399). A very significant calibration decision, as discussed earlier, is whether 
a loan will be offered directly by the government or through a private firm. While the latter is 
less resource intensive, it also carries a greater risk of adverse selection because banks face less 
severe consequences for lending imprudently. 


Government Insurance 


Government insurance is an economic tool by which the risk of suffering a financial loss is 
transferred from an individual, organization, or firm to a government agency. In exchange, those 
who transfer the risk (the insured) make small, regular payments called premiums to the govern- 
ment agency that takes on the risk (the insurer). If and when the financially damaging event 
occurs, the insured submits a claim to the insurer to cover the loss. The insurer takes on the risk 
of financial losses and receives premium payments from many insured clients at the same time. 
In this way, the risks of the group of insured clients are pooled, and the costs of adverse events 
are effectively distributed across all members of the pool. 

Public insurance programs differ from private insurance schemes in two significant ways. 
First, government insurance covers the kinds of risks that private insurers are often unwilling to 
cover, or coverage is offered to segments of the public who may not be accepted into private 
programs (Feldman 2002). Second, the goal of government insurance programs is not to earn a 
profit, but rather to achieve social and economic policy objectives. 

Government insurance is a low to moderately coercive instrument. Some programs are vol- 
untary, in which targets are enticed to participate by the prospect of having large financial 
liabilities underwritten by another party, whereas other programs are mandatory for everyone 
who meets certain criteria. Government insurance programs can also coerce targets to engage in 
activities that reduce their risk of making a claim. 

In the United States, for example, flood insurance is sold by the Federal Emergency Manage- 
ment Agency under the National Flood Insurance Program (NFIP). Its purchase is compulsory 
for property owners in designated flood areas who have a federally backed mortgage but is 
voluntary for all others. Premiums are broadly affordable, but only because they are deeply dis- 
counted, and the resulting loss of premium revenue has contributed to the NFIP’s debt to the 
US Treasury of more than $24 billion (Horn and Webel 2021). Federal flood insurance through 
the NFIP is available only to properties in communities that have adopted minimum floodplain 
management regulations that aim to limit development in flood hazard areas. 

The resource efficiency of government insurance programs depends on whether they are 
operated directly by the government or indirectly through a private firm. The latter design is 
less resource intensive because the skilled labor, sophisticated tracking systems, and data col- 
lection (all prerequisites for running solvent insurance programs) are not the purview of the 
government. Government insurance programs receive political support because they have the 
appearance of being self-financing, which is an attractive feature in neoliberal governance. This 
instrument also carries less political risk because it has low budgetary visibility. Premium income 
is documented in plain sight, whereas governments’ financial exposure is not. 

There are several critical calibrations that must be considered for designing government 
insurance programs. The first is the premium rate. Premiums can be set to cover program costs 
but can also be calibrated to convey a hidden subsidy to targets, as noted earlier. A second 
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design consideration concerns eligibility requirements and whether or not participation will be 
mandatory. Compelling participation is a significant advantage of public programs over private 
insurance because it allows for a large pool, which reduces adverse selection (whereby only those 
at significant risk opt for coverage) and achieves mutuality (i.e., a broad and diverse pool across 
which losses can be spread). 

A final calibration choice relates to whether participants should be compelled to undertake 
actions that reduce their chances of filing a claim. This can decrease the risk of moral hazard, 
meaning the perverse incentive to take risks knowing coverage is available, but can also impose 
inequitable costs on lower-income groups. Notably, achieving the optimal calibration for a gov- 
ernment insurance program requires huge amounts of data, sophisticated tracking systems, and 
the ability to react quickly to changing conditions — all of which are challenging for government 
providers. 


Fees and Corrective Taxes 


Fees and corrective taxes differ from the treasure tools discussed earlier in that they rely on nega- 
tive economic incentives to induce behavioral change. These instruments are often posited as an 
alternative to social regulation. Instead of compelling firms or individuals to stop a socially harmful 
activity by instating a ban, fees and taxes attach a financial penalty to the activity and allow targets 
to choose for themselves whether their continued participation is worth the additional cost. 

Fees are often imposed simply to raise additional revenue to pay for services, such as when 
there is a price for admission to public facilities. In other cases, however, costs are imposed to 
penalize socially undesirable behavior or attach a negative price signal to actions that undermine 
a government’s policy objectives. Special levies on products like cigarettes and alcohol — often 
referred to colloquially as “sin taxes” — are an example. Another example is corrective taxes, 
which aim to impose costs on specific target groups whose behavior contributes to social ills, 
such as higher property tax rates for homes located in risky areas that require more expensive 
protection from governments. There are few examples of such “risk-based” taxes in practice, 
presumably because they are vociferously opposed by those targeted by them. 

Fees and corrective taxes are moderately coercive (Cordes 2002). They impose financial pen- 
alties on undesirable activities but still leave individuals or firms the choice to either refrain from 
the activity or accept the cost. The resource efficiency of these tools is also moderate because 
the revenue they raise offsets some of the cost of their administration. Implementing fees and 
corrective taxes, however, involves a degree of political risk because they engender resistance 
among their targets. 

Fees and corrective taxes can be calibrated to suit the governance context and policy objec- 
tives. These instruments can be targeted reasonably precisely toward specific behaviors and 
actors. One essential calibration choice is the level at which the tax or fee is set. The rate must 
be high enough to deter the target behavior while not being overly suppressive. From a purely 
economic standpoint, the rate of the tax should be commensurate with the social costs of the 
activity; however, this is effectively impossible to calculate because of the volume and complex- 
ity of data that would be required (Cordes 2002). Some might also take issue with this calcula- 
tion entirely, arguing that there is no optimal level of a socially harmful activity. 


Tax Expenditures 


Tax expenditures encourage targets to engage in certain behaviors by reducing their tax obliga- 
tion. Its basic mechanism diverges from that of other instruments: instead of spending the tax 
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dollars it collects, the government forgoes revenue that it would otherwise be owed by individu- 
als or organizations. Tax expenditures are primarily discussed in reference to deductions from 
individual or corporate income tax bills, but tax expenditures can also be introduced for other 
classes of taxation, such as an estate or gift tax. A typical use of tax expenditure to promote 
climate change mitigation is write-offs for household energy efficiency transitions, such as the 
Crédit d’Impét Transition Energétique (CITE) in France (France 2021). Owners or occupants 
of a dwelling receive a tax credit for a percentage of what they spend on materials and equip- 
ment to limit the greenhouse gas emissions from their home. 

The tax expenditure instrument is low in coerciveness (Howard 2002). Targets are enticed, 
but not compelled, to engage in policy-preferred activities. Tax expenditure is a highly attractive 
instrument for policymakers because of its perceived resource efficiency: new tax credits and 
deductions can simply be added onto the existing tax system. However, the ease of administra- 
tion for this instrument tends to be overstated. Complex tax codes are intricate, subjective, and 
constantly changing, which causes immense frustration for both revenue agency employees and 
the public. Political support for tax expenditures is generally strong and bipartisan because it 
can be seen as both an extension and a contraction of government power. The tool is palatable 
to conservatives who are usually in favor of reduced taxes and is considered a reasonable com- 
promise by progressives who appreciate its ease of administration as compared to more direct 
instruments. 

When implementing the tax expenditure instrument, policymakers can decide between two 
forms. Tax deductions are amounts subtracted from the target’s total taxable income. Tax credits 
are amounts subtracted from the target’s tax liability. For tax deductions, the dollar value of a 
particular write-off is higher for people with greater amounts of taxable income, whereas for tax 
credits, the amount saved is the same regardless of income level. Another important calibration 
is the extent of targeting. More narrowly targeted programs are less likely to waste revenue but 
are more difficult to determine eligibility for. A weakness of the tax expenditure instrument is 
that programs are liable to grow as conditions shift and more people meet the eligibility require- 
ments. Such expansion often occurs without any explicit decision-making and can result in 
costly and ill-targeted programs. This problem is exacerbated by the fact that the mechanism 
lacks transparency, since tax expenditures are often difficult to see in the budget. 


Economic Instrument Mixes 


Policy instruments are rarely implemented in isolation. Rather, policy objectives are typically 
addressed using a combination of different instruments, called an instrument mix. Instruments 
can be mixed horizontally, whereby different tools are deployed by different administrative units 
of the same government, or vertically, when different levels of government, each with their own 
set of instruments, work cooperatively towards a shared policy objective. Mixes may be used to 
address different factors that contribute to the same problem, to support other tools in the mix, 
or to involve different governments in addressing the same overarching policy objective (del Rio 
2014). 


Instrument Mixes 


Instrument mixes can be described and assessed based on a variety of different criteria (Glaus 
2020). The first is the number of instruments in the mix. Older schools of thought in pol- 
icy studies have argued for parsimony when applying tools to objectives, purporting that the 
ideal ratio of instrument to goal is 1:1. This approach has a certain aesthetic appeal and would 
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theoretically minimize administrative costs, but this result is rarely borne out in the complicated 
realities of governance and decision-making. 

A second assessment criterion is the coerciveness and behavioral mechanics of the instru- 
ments in a mix. Economic instruments may be combined with one another or with instruments 
from other classes. There is some support for strategically combining more coercive instru- 
ments, such as corrective taxes and regulations, with less coercive instruments that encourage 
the desired behavior, such as subsidies and grants (van der Doelan 1998). The former, some 
scholars argue, are more capable of effectuating change but tend to generate resistance among 
their targets, which limits their usability. Resistance may be reduced by combining these tools 
with legitimating instruments that provide economic stimulus and reduce resistance toward the 
overall policy objective. These instruments may not be as directly impactful but are useful to 
lubricate the bargaining process with policy targets. 

A third feature of an instrument mix is its coverage of all theoretical points of application 
for the policy problem it addresses (Weber, Driessen, and Runhaar 2014). Any given policy 
problem is likely to have many different contributing factors or market failures, each of which 
provides a distinct opportunity to apply policy and effectuate behavioral change to achieve the 
policy objective. For example, the problem of the increased rate of sudden deaths during intense 
heat waves has numerous possible solutions. Cooling neighborhoods by planting trees, subsidiz- 
ing the purchase of air conditioners and fans, increasing the capacity of the health-care system 
to deal with acute heat during periods of high stress, and opening public cooling centers are all 
measures that could reduce the number of lives lost from extreme heat. 


Instrument Interactions 


A defining feature of an instrument mix is the interaction between its elements. A combination 
of two or more instruments may be ‘complementary’, wherein the instruments build linearly 
on one another toward accomplishment of the policy objective (Persson 2006). In ‘conflicting’ 
mixes, by contrast, instruments work against one another, potentially by invoking contradicting 
responses in the target, and they result in a smaller net impact than if only one tool had been 
used. The best designed mixes may even be ‘synergistic’, in which the instrument combinations 
enhance or legitimate one another, magnifying their overall impact. Instrument mix interactions 
can also occur on a broader level — mix ‘coherence’ occurs when there is integration of policy 
objectives and the specific goals of all instruments within a mix. 

Avoiding conflicts and achieving complementary, synergistic, and coherent mixes are, in part, 
a matter of design. More thoughtfully assembled mixes, built with strong coordination across the 
different governments or administrative units involved, may, for example, prevent the imposi- 
tion of conflicting criteria on a target. As an example, during some periods of the COVID-19 
pandemic, national and provincial governments in Canada each legislated quarantine require- 
ments for people who tested positive for the virus. In some cases, the requirements put forth 
by different levels of government contradicted one another, stipulating different durations of 
or restrictions on behavior. This resulted in confusion and reduced confidence in both sets of 
regulations (Ross 2021). 

Some tool conflict is inherent, because certain instruments simply have conflicting goals or 
involve a trade-off (del Rio 2014). In other cases, the success of an instrument is determined by 
the boundaries of the system for which it is assessed and will therefore look different when eval- 
uated by different levels of government with nested jurisdictions. Scholars have suggested using 
a multi-criteria framework to make explicit the trade-offs and congruences between different 
instruments in a mix. While not necessarily eliminating tensions between tools, this approach 
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allows for more deliberate and informed decision-making about the combination of instruments 
to be employed (Howlett and del Rio 2015). 


Deployment Over Time 


Instrument mixes are also characterized by their evolution and development over time. An 
often-cited principle of mix design is the scale of coercion in sequential instrument choices 
(Howlett 2019). Under the assumption that tools are technically substitutable, this maxim holds 
that governments will strive to first address a policy problem with the least coercive instrument 
available and reach for more coercive tools if and when they are necessary to overcome targets’ 
resistance to change. Following this principle, policymakers would first strive to achieve their 
goals through informing or educating the public before considering economic tools. If an eco- 
nomic instrument is ultimately deployed and it, too, fails to sway behavior, governments will 
only then resort to more authoritative tools, such as regulation. 

The reasons for this are described to be both programmatic and ideological: governments 
prefer tools that are inexpensive to implement and generate the least possible pushback among 
targets (Howlett and Rayner 2013, 173). The principle of increasing coercion has retained sig- 
nificant support, but there are many exceptions. The full possible sequence of instruments to be 
applied to a policy problem over years or decades is often not thought out in advance. Govern- 
ments may sometimes prefer to initiate pursuit of a policy objective with a strong-handed instru- 
ment to show their seriousness about tackling the issue. Then, once the initial hurdle is cleared, 
they are able to use less coercive instruments intended to sustain changes over time. 


Mix Design in Practice 


Implementation is critical to the success of any policy objective. No matter how well thought 
through a design may be, if the instrument mix is weakly implemented, the policy will fail (Rust 
1998). Effective policy designs must take into account the actual resources and capabilities of 
the implementing agents and strive for “goodness of fit” between the policy mix and the gover- 
nance context (Howlett and Rayner 2013, 9). Different implementing organizations will have 
their own ways of understanding and classifying the policy problem and modes of interfacing 
with the target population (Rist 1998). The goal of the implementing organization is also an 
important consideration. In some cases, the objectives or functions of an instrument may be 
misaligned with the agency that administers it, creating conflicting incentives. For example, tax 
expenditures are implemented by the revenue agency: a tool that functions by forgoing owed 
tax dollars is administered by the agency whose defining purpose is to collect revenue for the 
government (Howard 2002). This internal tension can complicate implementation of tax credits 
and deductions. 

Another key implementation consideration is the degree of freedom that policymakers have 
to develop new designs. The principles for instrument mixes described earlier imply that policy 
designers start from a blank canvas on which they are free to sketch their vision of an ideal 
tool combination. In reality, this is rarely the case. Decision-making about instrument mixes 
is almost always constrained by pre-existing policy agendas and tool portfolios (Howlett and 
Rayner 2013). As a result, design decisions about policy mixes more typically consist of incre- 
mental shifts that shape the function and objectives of a policy over time. This could consist of 
adding new instruments to a mix, deliberately retaining the elements of a mix despite changes 
to the policy context, or redeploying tools to serve different policy goals (Howlett and Rayner 
2013). 
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Considering the many constraints on decision-making, some have questioned whether poli- 
cies and instrument mixes can be meaningfully ‘designed’ at all (del Rio 2014). Policies are 
often driven more by opportunism and situational logics than by the principles described here. 
Additionally, the data supporting the various theories about instrument mixes is sparse. There 
is limited evidence demonstrating either the descriptive accuracy or the normative utility of 
these concepts for formulating policies in a complex governance environment. Consequently, 
literature on policy design is, perhaps, best framed as generating principles and frameworks that 
policymakers can orient toward. While many aspects are not applied to their full extent, aca- 
demic insight may support policymakers in deliberately designing the incremental changes that 
fall within the scope of their decision-making (Howlett and Rayner 2013). 


Limitations of Treasure Instruments 


Treasure-based instruments offer a diverse array of options for achieving policy objectives. Tools 
that employ financial incentives are less resource intensive than direct programming and can be 
precisely calibrated to promote change and leverage market efficiencies. These tools, however, 
also have limitations. Most importantly, the assumed mechanism for behavioral change implicit 
within economic instruments rests on a set of baseline conditions. Economic policy tools will 
successfully motivate change only if targets are driven by payoffs (Olejniczak, Sliwowski, and 
Leeuw 2020). This requires them to have the information and capacity to accurately calculate 
the costs and benefits of the target behavior. It also requires that this calculation is not distorted 
by the nature of the benefits being indirect or occurring in the too-distant future. For targets 
who do respond rationally to the incentives introduced by economic instruments, there are still 
hurdles to overcome before they adopt the policy-preferred behavior. Firms or individuals must 
also have the capacity (resources, knowledge, skills) and the opportunity (infrastructure, feed- 
back, social cues) to make changes (Olejniczak, Sliwowski, and Leeuw 2020). 

The policy design process is often criticized for holding unrealistic assumptions about human 
behavior and how it can be manipulated. The motivation, capacity, and opportunity for adopt- 
ing policy-preferred actions vary widely across target groups. “Target dependency” refers to 
how the effectiveness of economic instruments will depend on their targets’ financial need and 
receptivity to government funding (Howlett 2018, 129). For example, a crop insurance program 
that protects farmers from yield loss as a result of increased weather variability may appeal to 
wealthier farmers but have limited uptake among smaller operations that struggle to balance the 
books each season. Policy instruments may target individuals, households, or institutions. The 
behavior in focus may be a one-off, such as encouraging farmers to change their livestock to an 
indigenous breed that is better able to cope with variability in the local ecosystem, or repeated, 
such as changing the time interval for feeding one’s livestock to increase feed-use efficiency. In 
each case, different tools or calibrations may be required to effect the desired change. 

Inequity is another limitation shared by many economic policy instruments. Grants and sub- 
sidies perform poorly in terms of horizontal equity (Beam and Conlan 2002). The competitive 
application process for grants means that funds are not distributed according to need but are 
rather disproportionately allocated to those most apt at navigating the application process. Simi- 
larly, most subsidy benefits accrue to a minority of targets who are best able to situate themselves 
to take advantage. 

These instruments also present equity issues across time. Subsidies tend to provide the most 
benefit to those who take them up soonest, whereas those who enter the field after the subsidy 
ends lose out (Steenblik 2018). Government loans perform poorly in terms of vertical equity 
because they service only a narrow segment of the population in terms of socio-economic class 
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(those who are ineligible for private programs but still sufficiently creditworthy in the eyes of the 
government). Tax expenditures face a similar vertical equity issue as loans — the types of actions 
incentivized by these instruments tend to be achievable only for higher-income individuals. 
As a result, the majority of the benefits accrue to less needy segments of the population. Tax 
deductions are also not equitable across populations of similar income levels. This is because 
write-off amounts are not based on need but on whether a person meets particular criteria or 
has purchased a particular product or service that can be deducted. 

Finally, many economic policy instruments can have unintended effects on behavior or con- 
fer benefits on targets in a manner that does not further policy objectives. Windfall benefits 
occur when a policy target receives money for behaviors that they would have participated in 
regardless. The funds are not an incentive for change but are rather an unexpected gift. Subsi- 
dies, tax expenditures, and loans are all instruments that have the potential to waste public funds 
by providing windfall benefits. Relatedly, too-small economic incentives may have no effect 
whatsoever on their targets, making them highly inefficient (Rode, Gomez-Baggethun, and 
Krause 2015). 

Flawed assumptions about the mechanisms of behavioral change can result in responses to 
economic policy instruments that hinder achievement of the policy objective. For example, 
stormwater charges are intended to incentivize households to make landscaping changes to their 
property that will allow for more water to be absorbed into the earth and reduce flood risk. 
However, some property owners may perceive fee payment itself as “doing their part” to manage 
stormwater risks and crowd out pro-social motivation that may otherwise have been leveraged 
to incentivize policy-preferred behavior (Driver 2019). Worse still is the moral hazard associated 
with implementing government insurance. In each case, the behavioral response to the instru- 
ment works directly against the policy objective. 


Conclusion 


Treasure tools are a key governing resource for implementing policy objectives. Instruments that 
provide fiscal incentives and disincentives are more forceful than information-based policy tools, 
less coercive than social regulation, and involve fewer resources and less political exposure than 
direct programming. They can be precisely calibrated to meet policy objectives and, for some 
instruments, designed to leverage market forces for improved efficiency. Economic instruments 
are rooted in a conventional view of behavioral change: people will respond to stimuli rationally 
and in a manner that furthers their self-interest. 

However, as with any policy device, their actual implementation tends to complicate well- 
reasoned notions about efficiency and impact. Strong assumptions about the behavioral response 
and poor equity are common shortfalls of instruments in this category. These issues may be miti- 
gated by harnessing multiple kinds of instruments towards the achievement of a policy objective. 
Most economic tools also face some of degree of resistance rooted in the neoliberal aversion to 
public spending. Despite their risks and limitations, grants, insurance, loans, tax expenditure, 
and fees have been relied on to advance a broad array of policy objectives and remain an integral 
tool of governance. 
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INFORMATION AS A 
POLICY TOOL 


The Role of Government 
Communication 


Michael Howlett 


Government communication is now a large growth industry in many countries. Exactly 
what is meant by the term, however, varies from author to author. In this chapter, gov- 
ernment communication is conceived as a policy tool or instrument: that is, as a means 
to give effect to policy goals. Three key policy-relevant aspects of the term are exam- 
ined: (1) the link between government communications and the ‘nodality’ or information 
resource set out by Hood in his study of policy instruments; (2) the role of government 
communications in the ‘front-end’ of the public policy and production processes related 
to agenda-setting, policy formulation and producer activities as opposed to the ‘back-end’ 
of policy implementation, policy evaluation, consumption and distribution; and (3) the 
general aims of network management and overcoming information asymmetries which 
help explain the range of procedural and substantive policy tools used in government 
communication efforts. A model of four basic types of government communications is 
developed and examples provided of each general category. The implications of this analy- 
sis for cross-national comparative policy analyses of government communication activities 
and the evaluation of accountability and policy efficacy in contemporary governance are 
then discussed. 


Introduction 


Current governance modes have been shifting increasingly towards a pro-consultation mode 
which has led to the internalization and mandating of new communication practices in many 
jurisdictions (Feldman and Khademian 2007). These include the development and use of instru- 
ments which promote citizen empowerment such as freedom of information legislation, the 
use of public performance measures, various forms of e-government and the increased use of 
government surveys and advertising, among others. As a result, government communications 
is now a large growth industry in many countries and the subject of increasing attention from 
both practitioners and theorists. 
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Government communications are typically thought of as the ‘sermons’ in a ‘carrots, sticks 
and sermons’ formulation of basic policy instrument types (Vedung and van der Doelen 1998). 
Vedung, for example, has defined a ‘sermon’ as: 


Efforts to use the knowledge and data available to governments to influence consumer 
and producer behaviour in a direction consistent with government aims and wishes” 
and/or “gather information in order to further their aims and ambitions. 

(Vedung and van der Doelen 1998) 


The disparate nature of these government communication activities and the fact that many 
post-9/11 security-inspired government actions have also prevented information release, how- 
ever, are indicative of the need to be more precise in what is meant by the term ‘government 
communication’. Moves in the direction of increased consultation and information release are 
fraught, since moves in this direction involve trade-offs between some rights such as the public’s 
‘right to know’ and state security issues or an individual’s right to privacy. Any general diminish- 
ment of state power can be reversed in times of war or crisis, as has been the case in many coun- 
tries in the post—9/11 environment of the US-led ‘war on terror’, in which concerns with state 
and collective security in times of war or terrorism have led to a renewed emphasis on restricting 
information disclosure, as we have seen recently in many countries. 

Different foci make assessments and generalizations about trends and patterns of use exceed- 
ingly difficult (Ledingham 2003). Nevertheless. classifying and analyzing the wide range of 
activities and tasks that fall into the category of ‘government communication’ is an essential 
prerequisite for such assessments, which, in turn, are required if the contours and implications 
of new communications activity in areas such as the media and elections are to be understood. 


Defining Government Communication as a Policy Tool 


Exactly what is meant by the term government communication currently varies dramatically from 
author to author, ranging from its association with all forms of political activity (Deutsch 1963; 
Bang 2003) to a very specific focus on one limited type of activity, like political advertising (Fire- 
stone 1970; Young 2007). Needless to say, the consequences of the definition adopted greatly 
affect the conclusions reached pertaining to the growth and spread of communications activity 
and its impact. 

Thinking about such activities as ‘policy tools’, however, helps sort out the different goals 
and purposes of government communication, a first step towards establishing a typology of 
such activities, itself the first step towards effective empirical analysis and theory construction. 
That is, “government communication’ can be thought of as a generic name for a wide variety 
of a specific type or category of governing instruments, ones which typically draw upon what 
Christopher Hood (1986) called ‘nodality’, or the use of government informational resources 
to influence and direct policy actions through the provision or withholding of ‘information’ or 
‘knowledge’ from societal actors. 

As is well known, Hood (1983, 1986, Hood and Margetts 2007) argued that governments 
have essentially four resources at their disposal — nodality, authority, treasure and organizational 
(or ‘NATO’ in Hood’s terminology) — which they use to monitor society and alter its behaviour. 
In Hood’s scheme, instruments are grouped together according to (1) which of these resources 
they primarily rely on for their effectiveness and (2) whether the instrument is designed to effect 
or detect changes in a policy environment (Hood 1986; Anderson 1977). This formulation is 
useful in providing four clearly differentiated categories of policy instruments. While each tool, 
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to a certain extent, relies on all four resources (for example, a tax is a treasure tool but is also 
administered by an organization and relies on government authority to be collected and on 
the provision of information to taxpayers about its existence), in this scheme, each tool may be 
classified according to the primary resource it involves (in the case of a tax, this is ‘treasure’). 
However, for our purposes, this second distinction is less significant since communication is 
commonly two way, allowing a simpler taxonomy of basic governing tools and resources to be 
set out (see Table 27.1). 

Thus, in Hood’s taxonomy, most government communication activities can be seen to pri- 
marily involve the use of information-based policy tools, which function as they do because of 
the nodal position governments occupy in public policy systems and subsystems. This insight 
allows us to better describe and classify the various different techniques practiced by govern- 
ments in this area, to uncover patterns in their use and to begin to understand why these patterns 
exist. However, specifying the basic resource used by this tool is only the first step in the identi- 
fication of specific subtypes which differ in terms of how they control or manipulate knowledge 
and for what purpose(s). 


Distinguishing Between Different Types of Government 
Communication Tools 


Vedung’s definition is useful in this regard as it sets out two general dimensions of information 
tool use and the general purposes to which they can be put: influencing consumer and producer 
behaviour in economic transactions and controlling or collecting information for politico- 
administrative ones. That is, a primary distinction can be made between whether the commu- 
nication activities are intended to serve as devices primarily oriented towards the manipulation 
of policy actors and policy processes (Saward 1992; Edelman 1988) or social and economic 
ones involved in the production of goods and services (Hornik 1989; Jahn et al. 2005): that is, 
between their procedural and substantive use (Howlett 2000). 

Substantive instruments are used to alter some aspect of the production, distribution and 
delivery of goods and services in society, broadly conceived to include mundane goods and 
services (like school lunches) as well as a range of vices and virtues, ranging from crude vices 
(such as gambling or illicit drug use) to more common individual virtues (such as charitable 
giving or volunteer work with the physically challenged) to the attainment of sublime collec- 
tive goals (like peace and security, sustainability and well-being). We can thus define substantive 


Table 27.1 Examples of Policy Instruments, Classified by Principal Governing Resource Used 


Nodality/Information Authority Treasure Organization 
Information collection Command and control Grants and loans Direct provision of goods 
and release regulation and services and public 
enterprises 
Advice and Self-regulation User charges Use of family, community, 
exhortation and voluntary organizations 
Advertising Standard setting and Taxes and tax Market creation 
delegated regulation expenditures 
Commissions and Advisory committees Interest group creation Government reorganization 
inquiries and consultations and funding 


Source: Hood (1986) 
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government communication policy instruments as those policy techniques or mechanisms 
which rely on the use of information to directly or indirectly affect the behaviour of those 
involved in the production, consumption and distribution of different kinds of goods and ser- 
vices 1n society. 

Procedural tools, on the other hand, affect production, consumption and distribution pro- 
cesses only indirectly, instead affecting more directly the behaviour of actors involved in poli- 
cymaking. These actors are arrayed in policy networks which are comprised of very simple 
arrangements of nodes (actors) and links (relationships) but can result in very complex structures 
and interaction patterns. Policy networks include sets of formal institutional and informal rela- 
tional linkages between governmental and other policy actors which are typically structured 
around shared beliefs and interests in public policymaking and implementation. In order to pur- 
sue their preferred policy initiatives, governments must interact with other state and non-state 
actors who might possess diverging interests (Leik 1992). They use procedural communicative 
tools based on government information resources in order to attempt to alter the behaviour 
of policy network members involved in policy-making processes. They are only tangentially 
related to productive or consumptive behaviour, if at all. 


Distinguishing Between the Use of Government Communication 
Instruments at Different Stages of the Policy and Production Processes 


Making a distinction between procedural and substantive communication tools is a good first 
step in arriving at a reasonable taxonomy of such instruments which can inform empirical analy- 
sis and theory building. However, there is a second dimension which also requires elaboration: 
the stage of the production process or policy cycle upon which different communication tools 
focus (Howlett 2009). The different stages of the production process are well known and do not 
require further elaboration here. However the same cannot be said of the policy process. While 
different models of policy processes exist, historically, one of the most popular means for ana- 
lyzing public policymaking has been to think of it as a set of interrelated stages through which 
policy issues and deliberations flow in a more or less sequential fashion from ‘inputs’ (problems) 
to ‘outputs’ (policies). The resulting sequence of stages is often referred to as the ‘policy cycle’ 
(Jann and Wegrich 2007). In this model, agenda-setting refers to the process by which problems 
come to the attention of governments; policy formulation refers to how policy options are formu- 
lated within government; decision-making is the process by which governments adopt a particular 
course of action or non-action; policy implementation relates to how governments put policies into 
effect; and policy evaluation refers to the processes by which the results of policies are monitored 
by both state and societal actors, the outcome of which may be reconceptualization of policy 
problems and solutions. 

Many studies of procedural information tool use, for example, focus on the role of govern- 
ment communications as part of the agenda-setting process in government (Mikenberg 2001; 
Sulitzeanu-Kenan 2007) or on its role in policy implementation (Salmon 1989a, 1989b). In the 
case of production processes, the focus has also been upon specific stages of production pro- 
cesses, such as the effort to affect consumption activities and actors or those involved in produc- 
tive or distributive activities. These are all quite different roles, however, and they also should 
be carefully distinguished from each other in order to understand the links that exist between 
government communication strategies and activities and policy outcomes such as accountability 
and policy efficacy. 

In general, substantive communication tools can be focused primarily either at the ‘front end’ 
or production stage of the provision of goods and services or at the ‘back end’ or consumption 
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and distribution stages. Similarly, procedural tools can also be focused on different stages of the 
policy process, as depicted in Table 27.2. 

Government communications typically do not affect decision-making per se, but they are 
directed at all other stages of the policy process. 


The Four Basic Types of Government Communication Tools 


This discussion suggests that government communications tools can be thought of as falling into 
four general types (see Table 27.3). As Hood (1986) noted, all policy tools can be targeted at 
different levels of society. In the case of information-based tools, Adler and Pittle (1984) have 
suggested a division along the lines of targeting individuals, groups and populations as a whole. 

The most high-profile and thus most commonly observed and chronicled type of tool is the 
substantive, back-end tool focused on the effort to alter consumer behaviour: the government 
information campaign. This includes various campaigns waged by governments to encourage 


Table 27.2 Five Stages of the Policy Cycle and Their Relationship to Applied 
Problem Solving 


Applied Problem Solving Stages in Policy Cycle 


Problem Recognition Agenda-Setting 


Proposal of Solution Policy Formulation 
Choice of Solution Decision-Making 


Putting Solution into Effect Policy Implementation 


WP One 
Ce Oy eS 


. Monitoring Results Policy Evaluation 


Table 27.3 Four Categories of Government Communicative Tools 


Policy Purpose 


Substantive Procedural 


Stage of Policy Cycle/ Front end Notification instruments/ General information 


Production Process moral suasion 


(agenda-setting 


disclosure or prevention 


Primarily Targeted and policy (e.g., consumer product (e.g., freedom of 
formulation/ labelling, prospectus information and privacy 
goods and disclosure laws, laws, performance 
service government e-health and measures, censorship) 
production) e-government portals, 

appeals to producers 
with or without the 
threat of regulation) 
Back-end Exhortation and Data collection and release 
(policy information campaigns (e.g., censuses, compulsory 
implementation (e.g., moral suasion and reporting, press releases, 
and policy government advertising) media relations, 
evaluation/ goods government websites) 


and service 
distribution and 
consumption) 
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citizens to, for example, eat well, engage in fewer vices and otherwise behave responsibly. How- 
ever, ‘front-end’ communication activities aimed at altering producer behaviour through pro- 
vision of product and process information to customers (‘product information’) are also very 
prominent. Most product-related labelling and other such activities fall into this category, as do 
most government e-services which are designed to encourage producers to be better aware of 
laws and regulations or to produce particular types of goods and services and not others. 

The most commonly observed and chronicled category of procedural tool is the front-end 
procedural category which focuses on the use of general information prevention or disclosure laws 
and other tools — such as access to information laws — in order to provide policy network actors 
with the knowledge required to effectively filter and focus their demands on government for new 
policy measures or reforms to older ones. However, governments are also very much involved in 
the use of communications to promote efficient policy implementation and positive policy evalu- 
ations, through the use of data collection and release tools such as media releases, data collection, 
surveys and, increasingly, government websites (Gandy 1982; Hood and Margetts 2007) to provide 
additional information to policy network members in specific sectoral or issue areas. 

Each of these four general categories of communications tools is discussed next and some 
examples of their use in Canada provided. 


Substantive Producer-Directed Tools: Notification Instruments 
and ‘Moral Suasion 


Adler and Pittle (1984) describe this tool as ‘notification instruments’ which 


convey factual information to the intelligent target. Implicit in the notification 
approach is the belief that the target, once apprised of the facts, will make the appro- 
priate decision. 


Some notification tools attempt to be purely factual, ongoing and passive in nature, such as 
nutritional labelling on foodstuffs or health warnings on cigarettes (Padberg 1992; Baksi and 
Bose 2007). They are usually enacted in regulations (i.e., disclosure is mandatory) and are aimed 
at providing information to consumers allowing them make better decisions or overcome infor- 
mation asymmetries (Jahn et al. 2005) between producers and consumers, with the expecta- 
tion that they will change their behaviour in some way consistent with government goals — for 
example, reducing smoking or eating nutritional foods. All these activities are intended to have 
an effect on producers and production decisions: for example, manufacturing fewer tobacco 
products or producing healthier foods. 

Similarly Stanbury and Fulton (1984) describe ‘moral suasion’ as a more direct plea from gov- 
ernments to producers ‘whereby voluntary action is urged under threat of coercion if refused’. 
Many countries administer important aspects of their financial systems in this fashion: for exam- 
ple, asking banks, taxpayers and other financial institutions to act in a certain way (e.g., keep 
interest rates low or allow certain groups to borrow funds) with the implicit or explicit threat of 
direct government regulation if requests are ignored or go unfulfilled (Bardach 1989). 


Substantive Consumer-Directed Tools: Exhortation 
and Persuasion Instruments 


Adler and Pittle (1984) provide a definition of substantive, information-based, ‘back-end’ tools 
directed at consumers as ‘persuasion instruments’ which entail ‘persuasion schemes [which] 
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convey messages which may or may not contain factual information which overtly seek to moti- 
vate target audiences to modify their behaviour’ (Adler and Pittle 1984). 

These are probably the best-known government communication tools. The most prominent 
type is the appeal from political leaders to various social actors, urging them to follow a gov- 
ernment’s lead in some area of social or economic life (Cobb and Elder 1972). Stanbury and 
Fulton (1984) describe ‘exhortation’ as ‘pure political leadership such as appeals for calm, better 
behaviour, and high principles’. 

Mass media and targeted information campaigns are also highly visible examples of tools in 
this category by definition and tend to be aimed less at producers than at consumers. Some of 
these information campaigns are more active and less factual but have the same intent: that is, 
providing social actors with more information about aspects of their behaviour and its advan- 
tageous or deleterious quality, urging enhancement of the former and a diminishment of the 
latter. Such campaigns are often conducted at the mass level and use a variety of mass-media 
delivery mechanisms (commercials, broadcasts, newspaper advertisements and the like). High- 
profile campaigns in many countries to prevent drinking and driving or encourage the purchase 
of victory bonds during wartime are good examples of this kind of instrument. 

Mass campaigns began with the emergence of mass media and are now common in most 
countries. The information often transmitted through information instruments is not always so 
factual, however, but can be used to ‘sell’ a government’s policies in the same way that other 
products are marketed. Many national governments are now the largest purchasers of advertising 
in their countries and far outstrip national brands well known for their advertising overkill, such 
as alcoholic beverage and soft drink companies, as well as fast food chains. 

The federal government of Canada, for example, has been the largest advertiser in country 
since 1976 (Stanbury et al. 1983), with the larger provincial governments in the top ten as well. 
Ryan (1995) noted that federal advertising expanded from $3.4 million in 1968 to $106.5M in 
1992, a 3000% increase. Even inflation adjusted, this amounted to a 665% increase in 25 years. 

Specific national issue campaigns can be very costly. Alasdair Roberts and Jonathan Rose 
(1995), for example, conducted an in-depth study of a mass media campaign conducted by the 
federal government of Canada to introduce a new goods and services tax in 1989—1990. They 
found the federal Department of Finance to have spent $11.6 million on public education in a 
combined print/radio/TV campaign, $5 million on direct mail materials and $5 million on a call 
centre; Revenue Canada (Customs) to have spent $10.6 million on advertising and $9.2 million 
on instructional material; and Revenue Canada (taxation) to have spent a further $28 million 
advertising a GST credit while a specially created GST Consumer Info Office spent $7.4 million 
on advertising and $6.9 million on production, The total for this one campaign was $85 mil- 
lion. This was more than the largest private sector advertisers spent in all of 1989. For example, 
Proctor and Gamble, with its hundreds of consumer products, had a total advertising budget of 
$56.7 million. 


Procedural Tools Affecting Agenda-Setting and Policy Formulation: 
General Information Disclosure and Concealment 


Stanbury and Fulton (1984) describe common ‘front-end’ procedural policy tools as monitoring 
and information disclosure instruments (for example, environmental audits, ombudsmen, prices 
and incomes commissions, as well as freedom of information and privacy laws). All these tools 
can involve information bans or release prevention as well as information disclosure; they can be 
general or specific in nature, can be focused on individuals or the public and can be mandatory 
or optional in nature. 
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Freedom of information legislation allows access to individual’s own records and those of 
others with numerous exemptions — again many benign (to protect other individuals from 
unnecessary disclosure) — and access to information, allowing access to documents and records 
of others with numerous exemptions — again, many benign (to protect individuals from unnec- 
essary disclosure, privacy rights). These legislative arrangements were a feature of the Scandina- 
vian system of ombudsmen for administrative control and were introduced in many countries in 
the 1970s and 1980s (Relyea 1977; Bennett 1988, 1990, 1991, 1992; Bennett and Raab 2003; 
Bennett and Bayley 1999; Howe and Johnson 2000). Whistleblower acts are an extreme example 
of the use of communication tools focused at the individual level. They are bills intended to 
protect people who speak out about problems in the government’s bureaucracy. Through such 
legislation, bureaucrats are often offered legal protection against reprisals for reporting govern- 
ment wrongdoing. 

There is also a wide range of these tools designed to protect certain kinds of information on 
government activities or in government files from entering the public realm or to suppress cer- 
tain knowledge or information considered undesirable. These include protecting not only infor- 
mation collected by governments but also that which comes into their possession (for example, 
from foreign government or via documents filed in court cases etc.). These range from wartime 
(and peacetime — e.g., the Ontario Film Censors Board) censorship and bans on political parties 
and speech (e.g., hate crimes legislation etc.) to Official Secrets Acts (as in the UK), with various 
levels of confidentiality and penalties imposed for publicizing or releasing government secrets. 

Censorship has occurred in many countries during wartime but also in peacetime (e.g., 
film and theatre censorship). This latter use has been slowly whittled away as individual rights 
in democratic states have been ruled to trump government or collective ones (Qualter 1985). 
However, Official Secrets Acts are the most important statute relating to national security in 
many countries and are designed to prohibit and control access to and disclosure of sensitive 
government information (Pasquier and Villeneuve 2007). Offences tends to cover espionage and 
leakage of government information. The meaning of the term official secrets varies dramatically 
from country to country but broadly allows governments to classify documents and prohibit 
release of different categories for sometimes very long periods of time (e.g., 50 to 75 years). 


Procedural Tools for Policy Implementation and Evaluation: 
Data Collection and Release 


Stanbury and Fulton (1984) also discuss several tools in which affected parties are given informa- 
tion on government plans, like public hearings or the discreet use of confidential information 
such as planned leaks to press or planned public disclosure of government intentions as well as 
government media relations and communications strategies intended to legitimize government 
actions and pre-empt criticism and dissent. 

Some of these tools are used to generate information, such as inquiries, surveys and polling. 
Government information requests can be very focused and can be quite secretive (for example, 
in the immediate aftermath of the 9/11 airline hijackings, when the US government urged credit 
card companies to provide records of suspicious activities by suspected hijackers). Benchmarking 
and performance indicators are more visible but similar tools involving the use and publication 
of indicators of government and non-governmental performance designed to collect and release 
information on specified activities of organizations against set written standards (Benjamin 2008; 
Sharma and Wanna 2005). Of course, there are problems in both the private and the public sec- 
tor in terms of agreeing what to measure and how to do so (for example, whether to measure 
inputs or outputs) which have limited their spread (Papaioannou and Bassant 2006; Johnsen 
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2005; Adcroft and Willis 2005; de Lancer and Holzer 2001; Van Dooren 2004; Cohen and 
Santhakumar 2007). Many recent government efforts at e-communications (websites, mail lists, 
wikis, Twitter and the like) also fall into this category. 

Tools in this category can also be used to prevent specific types of information from circulat- 
ing. Privacy acts, for example, exist in many jurisdictions as a counterpoint to access to infor- 
mation laws in which specified information is excluded from such acts. Some jurisdictions have 
specific legislation devoted to this subject, usually with a focus on protecting personal informa- 
tion in areas such as health, financial and tax matters and with respect to criminal proceedings. 


Conclusion: General Comments and Patterns of Use 


As has been set out in this chapter, there are many different kinds of government communica- 
tion activities, and the lack of an effective taxonomy or framework for their analysis has made 
generalizing about their impact and patterns of use quite difficult. Conceiving of such activities 
as information-based policy tools helps highlight the similarities and differences between dif- 
ferent instruments and helps develop a relatively parsimonious taxonomy of their major types, 
which can facilitate national and cross-national studies of their use and impact. 

Distinguishing between information tools that are procedural and substantive in nature is a 
first step in the development of this taxonomy. The second step is to distinguish between those 
procedural and substantive instruments that focus on the front-end of policy and production 
processes and those which focus on the back-end. Taken together, these two criteria reveal four 
distinct types of communication instruments: product information, consumer information cam- 
paigns, general disclosure tools and data collection and release tools. 

Classifying communications instruments in this way is the first step towards the develop- 
ment of empirical assessments of the rationale for their use and uncovering any patterns of 
their employment, both spatially (cross-nationally) and temporally (historically). Hypotheses 
suggested in the literature, for example, include the arguments that information instruments will 
tend to be used only in situations in which: 


1. 100% compliance is not required for a policy to be effective. 

2. Government and public interests coincide (e.g., on health awareness) so that government 
appeals are likely to be favourably received. 

3 The crisis situation is relatively short-term (Rose 1993), when other tools may require too 
much lead time to be effective. 

4. Itis otherwise difficult to impose sanctions. 

5. The issue in question is not very complex (technological or legal) in nature but can be 
reduced to the level of advertising slogans (Romans 1966; Vedung and van der Doelen 
1998). 
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NON-STATE MARKET-DRIVEN 
TOOLS 


Capacity Challenges in Non-State 
Governance Contexts 


Benjamin Cashore 


After a quarter century of design tinkering and impacts, students and practitioners of “non-state 
market-driven” NSMD policy tools face two related empirical conundrums: first, most of the 
problems for which they were offered as more effective and efficient tools have become more, 
not less, acute, and second, these tools continue to proliferate among global and private gover- 
nance arenas. We argue that reversing course requires three related steps. First, NSMD policy 
tools must be designed based on the features of a specific problem at hand. This requires both an 
understanding of the features of the problem — be it climate or species loss — and interrogation of 
expected inverse effects on other problems. Second, careful assessment of the way in which these 
tools have been used as a political devise to change problem definitions or minimize regulatory 
constraints must be incorporated into the design. This requires incorporating analytical capacity 
more consciously into the design, rather than relying solely on policy and management capac- 
ity that may miss these dynamics. Third, policy designers must focus not only on evolutionary 
pathways through which NSMD tools might be designed to travel but also on their ability to 
indirectly diffuse to improve the effects of public policy processes. 


Introduction: Non-State Market-Based (NSMB) Policy Tools 


Non-state market-based tools are third-party certification tools which emerged a generation 
ago to address particular policy problems, such as deforestation, fisheries depletion, land deg- 
radation, and climate emissions. These tools create a set of standards or criteria for product or 
production processes — such as to sustainably harvest aquaculture or mariculture products or not 
to harvest old-growth timber — and a third-party organization which issues a certificate or stamp 
of approval to particular products or producers for adhering to these standards. These certificates 
can then be important determinants of further supply-chain issues, such as when a hardware or 
grocery store agrees only to stock and sell certified timber or salmon, as the case may be. 

The origins of the first full-fledged NSMD governance system can be traced to the con- 
cerns of stakeholders around global forest degradation in the 1980s. Environmental groups, 
social activists, and even likeminded businesses were frustrated with the array of governmental, 
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intergovernmental, and non-governmental initiatives that were designed to address deforesta- 
tion and degradation and promote sustainable forest management. Environmental groups were 
concerned that the International Tropical Timber Organization’s (ITTO) approach to promot- 
ing forest management practices and trade in the tropics (Dauvergne 2001; Gale 1998) simply 
created a “logging charter” and were disappointed that the 1992 Rio Earth Summit (UNCED) 
(Bernstein and Cashore 2004; Humphreys 1996) failed to achieve a binding global forest con- 
vention. While many saw potential in tropical timber boycott campaigns, there was a recogni- 
tion that these were short lived and blunt and often fostered conversion to less scrutinized crops. 

The Forest Stewardship Council’s (FSC) solution was to develop a broad mission to improve 
the management of the world’s forests (Synnott 2005), which would house and simultaneously 
promote a range of social and environmental objectives. As a result, the broad coalition of envi- 
ronmental groups, social activists, and business supporting FSC championed different issues, 
including tropical forest loss and degradation, indigenous and forest dwellers’ rights, and sustain- 
able forest management generally (Rametsteiner and Simula 2003; Siry, Cubbage and Ahmed 
2005). FSC’s governance arrangements soon adopted a tripartite model that established three 
evenly weighted voting chambers: social, economic interests, and environmental. Votes in each 
chamber are weighted evenly between developing and developed countries. FSC mandates the 
establishment of national or regional initiatives, which are charged with drafting locally appro- 
priate standards consistent with FSC’s global Principles and Criteria (P&C). 

The FSC approach sparked a myriad of non-state market-driven certification programs 
designed to address a range of problems from fisheries depletion to climate emissions, mining 
impacts, agriculture, coffee, and tourism (Auld 2014; Auld, Balboa, Bartley et al. 2006; Auld, 
Balboa, Bernstein et al. 2009; Auld and Cashore 2012a, 2012b; Auld, Renckens and Cashore 
2015; Bozzi, Cashore, levin et al. 2012; Kanie, Haas, Andresen et al. 2013; see also Vince, 
Chapter 8 in this volume). Cashore (2002a) identified (Table 28.1) five key features of NSMD 
certification systems that have been modified over the last two decades by Cashore, Auld and 
Newsom (2004a), Bernstein and Cashore (2007a), and Auld and Cashore (2012a). 

This review highlights that the most significant capacity challenges for realizing their prob- 
lem-solving potential is the ability to reflect on and theorize about forward-thinking strate- 
gies and collaborations. Such a reorientation, as Bernstein et al. (2000) argued over a decade 
ago, challenges most empirical social science focused on explaining empirically measurable 


Table 28.1 NSMD Governance: Key Features 


National/Global Policy standards (settings/specifications) must be consistent with global criteria 


Dimensions (objectives) and principles (goals) 
Collaborative Government actors excluded (deemed ineffective as governing arena) 
Governance Business limited (must share equally with environmental and social interests) 
Practice: Emphasis on clear “on the ground” prescriptive requirements about specified problems 
Rules 
Practice: Tend to be broad 


Policy Scope Labor 
Indigenous rights 
Wide-ranging environmental impact 

Authority: Emergence phase: consequentialist (market incentives for those being certified) 
Entrenchment phase: appropriateness (political legitimacy) 


Source: Adapted from Cashore (2002a) and Auld and Cashore (2012a, 2012b) 
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phenomena that have already occurred to also focus on theorizing about the contingent empiri- 
cal pathways on which we are embarked and through which strategic interventions shape and 
influence pathways. 


Capacity Challenges in NSMD Tool Use 


NSMD tools are a form of “co-production” in which producers, wholesalers, retailers, and oth- 
ers work together with certifiers to create, monitor, and enforce standards. Although often built 
on the threat of further government regulation, they are “non-state” in the sense that rules and 
standards are generally not codified in law — although they may be referenced by them — and 
public officials also generally are not involved in enforcement activity related to them. However, 
it is certainly the case that governments can promote or discourage the creation of NSMD-type 
certification regimes, sometimes successfully and sometimes not. 

Just as with more traditional state-based policy tools, NSMD do require certain competences 
and capabilities on the part of companies and third-party certifiers if they are to survive and/ 
or work effectively, but also require competent and capable public officials if they are to remain 
legitimate and “authoritative” and, for example, are not to be replaced by more traditions forms 
of state-based standards and sanctions. 


Capacity Challenges in Policymaking 


One of the most important issues for students of public policy and management in the global era 
is to identify critical capacity gaps through which government officials might foster collaborative 
processes that help uncover complex instrument mixes for furthering, rather than exacerbating, 
identified policy goals. Drawing on Howlett and Ramesh (2016), three overall capacity require- 
ments can be identified which are generally relevant to collaborative policymaking, the impor- 
tance of which varies across different kinds of collaborative arrangements. 

Political acumen capacity refers to individuals who are aware that a “keen nose for politics 
not only within an organization but also the broader environment is essential for policy actors 
to be able to play an effective role in the policy process” (Howlett and Ramesh 2016: 306). This 
requires individual policy actors to develop a sound “understanding of the key stakeholders, 
their key interests, and their strategies and resources” (ibid.). 

Policy analytical capacity focuses on the “skills [policy actors] possess for diagnosing problems 
and developing appropriate strategies for addressing them,” including the ability to (1) “acquire and 
use internal and external knowledge”; (2) “access and apply technical and scientific knowledge 
and analytical techniques”; and 3) “absorb and process information or evidence in recognizing, 
formulating, deciding, implementing, and evaluating policy” (Howlett and Ramesh 2016: 305). 

Managerial expertise capacity emphasizes “the ability to perform key managerial functions — 
such as planning, staffing, budgeting, and directing.” These technical skills are given much atten- 
tion in management curricula and include “communication skills; leadership; teamwork; 
budgeting and financial management; decision making and problem solving; and ethics and 
integrity” (ibid.). Howlett and Ramesh emphasize leadership skills, as “especially critical if 
groups are to assume new challenges and devise new strategies for meeting them” as well as 
those involving “budgeting, accounting, and human resource management” (ibid.). 

These capacity challenges are key for realizing the potential of a range of collaborative gover- 
nance approaches including consultative in-house delivery, contracting out, commissioning, co- 
management, and co-production, each of which has its own “Achilles heel” capacity challenge. 
(See Kekez et al. 2020. See also Howlett and Ramesh 2016). 
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Arguably, however, no public collaborative mode requires more careful attention to these 
capacity deficits and pathways through which collaborative governance mechanisms must navi- 
gate than do multi-stakeholder certification systems. 

This is because, as modes of non-state market-driven (NSMD) governance, policy actors are 
presented with an additional hurdle from other collaborative forms: 1.e., how to help build, rather 
than maintain, legitimacy and authority to govern (which requires political acumen capacity) 
and, related, an ability to assess what types of problems and associated historical processes are best 
addressed and unleashed through NSMD governance (which requires both political acumen and 
policy analytical capacity). 

These skills are important because, as we show next, they permit government officials to 
develop strategies that can help institutionalize legitimacy and authority which, once achieved, 
will change impacts and problem-solving potential of the instrument or mix in question. Traveling 
such pathways requires that government officials undertake deliberate theoretical and conceptual 
thinking in which a number of possible futures are envisioned, rather than relying solely on back- 
ward-looking evidence that biases analysts to (often poorly conceived) experiments “already run”. 

This is important since, as we review here, application of political acumen and policy ana- 
lytical capacity reveals not one, but four “problem-solving” pathways (Cashore 2016; Cashore, 
Matas and others 2012) through which certification systems might have influence. We argue that 
while the ability of government officials to travel any one of the four pathways requires political 
acumen and policy analytical capabilities, specific types of managerial capacities will vary some- 
what depending on which pathway is chosen. 

Recognition of this leads us to modify Howlett and Ramesh’s (2016) “Achilles heel” argu- 
ment by positing a hierarchical pecking order through which the three policy capacities must be 
applied if NSMD certification as collaborative governance is to realize its potential. Political acu- 
men must be first exercised; policy analytic must be second; and managerial, while fundamental, 
must be applied subsequently. While this order is plausible based on historical evidence, we 
argue that the vast majority of skills held by government officials and non-governmental organi- 
zations seeking certification systems are not consistent with this pecking order. That is, manage- 
rial capacity has been prioritized, which has undermined the analysis of historical trajectories 
and much more complex arrangements, through which government officials can help nurture 
and navigate certification systems. This, in turn, has at times led government and NSMD offi- 
cials to overly rely on them to address too many public policy challenges which has, ironically, 
led to them underperforming (van der Ven and Cashore 2018). While lessons from this review 
can be applied to organizational and system levels (Howlett and Ramesh 2016: 304), we focus 
our attention on the types of political, policy, and managerial capacities and capabilities required 
of individual government officials. 

We elaborate these arguments in the following steps. First, we earlier identified the key fea- 
tures of global certification systems that structure how collaborative governance pathways are 
to be discovered and collectively traveled. Second, we review how political acumen and policy 
analytic capacity are key for government officials if they are to uncover and influence not one, 
but four potential pathways though which influence on public problem solving might occur. 
Third, we review the managerial capacities that government officials would need from the 
application of these higher-level capacity challenges, revealing how some are required regardless 
of the pathway, while others are unique to specific ones. We conclude by reflecting on how to 
fill and prioritize political and policy capacities gaps such that government managers can help 
nurture, rather than undermine, potential problem-solving pathways. For clarity, we focus our 
collaborative certification example on efforts to regulate forestry as the most advanced case of 
NSMD globally. 
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Direct and Indirect Evolutionary Pathways: Political Acumen 
and Policy Analytic Capacities 


The first task facing government officials who seek to draw on collaborative governance to 
address problems, usually the domain of government actors, is to identify how to improve sup- 
port in the marketplace for certification systems — be it in the form of consumer “willingness to 
pay” to decisions by firms along global value chains, often owing to NGO targeting campaigns 
— to reward and provide benefits to producers who agree to abide by the private regulations. 
Such exercises require a high degree of political acumen capacity because officials must be able 
to assess the broader context in which certification as collaborative governance has emerged. 
This includes an ability to understand the historical role of different stakeholders, power dynam- 
ics, and problem definitions they seek to nurture, from environmental challenges such as endan- 
gered species and biodiversity conservation to the economic role of forests for improving the 
plight of impoverished peoples. Political acumen capacity 1s also key for understanding not just 
geo-politics, but the contribution of different explanatory frameworks for observed support and 
opposition as they are key for identifying future courses of action (Bernstein, Lebow, Stein et al. 
2000), including triggering shifts in motivations over time. 

At the same time, policy analytic capacity is fundamental since collaborative certification 
systems do not just build new global institutions governing through the marketplace; they also 
attempt to address specified problems that expand on traditional regulatory approaches (Auld 
2014) (Auld, Balboa, Bernstein et al. 2009; Auld, Cashore and Renckens 2014) by assessing 
how standards might nurture, rather than straightjacket (Greening and Gray 1994), collaborative 
governance potential. Policy analytic capacity is also required for reflecting on how technical/ 
managerial decisions can be linked to, rather than undermine, the broader project of building 
legitimacy and authority (Auld, Cashore, Balboa et al. 2010) (see Table 28.2). 

Looking over 25 years of research and practice, the role of political acumen and policy 
analytic capacity was fundamental in uncovering pathways in which norms and preferences 
changed in response to strategic interventions. These capacities were key in conceptualizing not 
one, but four evolutionary pathways through which impact might occur. Where these capaci- 
ties were absent, government officials and strategists often undertook well-intended efforts that 
stymied their potential. Hence, policy actors capable of intervening and reflecting on future 
impacts tend to have the political acumen capacity that comes from training in historical insti- 
tutionalism (Thelen 2003) and comparative historical sociology (Mahoney 2008) in ways that 
complement, rather than being subordinated by, those fields that emphasize backward-looking 


Table 28.2 Individual Policy Capacity Requirements for Identifying and Traveling Causal Pathways: Cross 


Cutting 
Type Tiaining 
Political Sophisticated social science training in international relations, economics, sociology, 
Acumen organizational and institutional analysis that emphasizes norm change, historical 
trajectories, and transformative triggers 
Policy Domestic, comparative, and international policy process, policy sciences, stakeholder 
Analytical coalition building including role of policy learning mechanisms, internal and external 
perturbations 
Managerial Excellent communications, budgeting, leadership and team building skills, cost benefit 
analysis 
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‘evidence-based’ (Sanderson 2002) and ‘data-driven’ analysis that, taken in isolation, ironically 
reinforced the priority of managerial capacity. 


Direct Pathway 


The direct pathway is the one that the vast majority of scholarly and practitioner work on 
NSMD political acumen and policy analytic attention has been placed. Scholars who emphasize 
an evolutionary approach, including my own collaborative work, have posited that if NSMD 
systems gain authority to directly create policy to which most businesses adhere, strategists must 
confront a conundrum inherent in the design of the system: the higher the standards initially, the 
lower the support, and only modest impacts could be expected; conversely, the lower the stan- 
dards, the higher the support, but only modest impacts would accrue (Cashore, Auld, Bernstein 
et al. 2007a) (see Table 28.3). 

In other words, practitioners with these types of capacity skills have spent a great deal of time, 
as have researchers, on identifying these ‘causal influence logics’ that might help foster coherent 
choices by businesses, environmental groups, and social activists — and even governments — and 
might, through nurturing collaborative governance arrangements, ultimately achieve “high sup- 
port, high standards and high impacts” (Bernstein and Cashore 2007a). 

The strategic implications that emerge from the application of policy analytic capacity are 
profound: supporters must develop certification standards that are high enough to be meaningful 
for addressing an “on the ground challenge”, but at a level that rewards existing “top producers”, 
who are often practicing at a high level owing to government regulations. Application of this 
lens has practical significance in understanding historical efforts that both applied and did not 
apply these policy acumen insights. 

For example, a generation ago, well-intended environmental activists worked to increase 
FSC standards in British Columbia, Canada (Cashore, Auld and Newsom 2004b), following 
widespread interest on the part of most industrial players to support FSC. However, these activ- 
ists failed to analyze carefully why firms were supporting FSC — which was largely owing to 
increases in government-initiated regulations. Therefore, the change logic was that not sup- 
port for the FSC would enhance domestic forest practices in highly regulated jurisdictions, but 
rather, in drawing on support from firms operating in these settings, greater market signals might 
be generated to target the behavior of firms operating in less regulated jurisdictions. 

However, lack of capacity to apply this type of thinking resulted in most BC firms vacating 
their original support and turning to the much less prescriptive industry standard. Similar results 
also occurred in the Canadian Maritimes (Cashore and Lawson 2003) and Finland (Cashore, 
Egan, Auld et al. 2007) FSC standard-setting processes. Following these institutional and policy 
failures, FSC and its supporters began to fill political acumen and policy analytical capacity gaps, 
which led to strategies to foster standard-setting processes elsewhere in Canada in the boreal 


Table 28.3 The Dilemma of High and Low Certification Requirements, Time 1 


Requirements of Certification Systems 


High Low 
Level of Firm Support Low High 
Impacts on Sustainability Low Low 


Source: Cashore, Auld, Bernstein et al. (2007b) 
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Table 28.4 NSMD Direct Pathway 


Causal Influence Logic Strategic Implications 

Must achieve strong support Strategists must gain support from most producers to be effective 
among purchasers along Standards should be set to reward, not punish, firms that are already 
transnational supply chains highly regulated 


Increases in standards must follow, not precede, enhanced market uptake 


forests (Auld 2006), more consistent with the governmental regulation “as spark” trajectory. 
During these unfolding historical processes, public managers with policy and analytic capacity 
were best positioned to nurture collaborative certification systems that fostered public agendas. 
For instance, those seeking to draw on certification to foster domestic sustainable forest manage- 
ment goals could envision the efficiency and international credibility gains for supporting these 
efforts, such as certification-friendly procurement policies, to providing resources for fostering 
collaborative certification dialogues (Cashore 2002b: 510), rather than viewing them solely as a 
threat to state sovereignty. 

Application of this type of political and policy acumen capacity led a number of scholars 
and practitioners to reflect on the ways in which collaborative governance that includes highly 
different motivations among business, environmental and social activists, might shift eventually 
toward collective norms about the appropriateness of the institutional arena (Bernstein and 
Cashore 2007a). These efforts have taken up by integrating Bernstein and Cashore’s work on 
international pathways with a focus on policy learning regarding strategy and institutional for- 
mation (Cashore, Leipold, Cerutti et al. 2016; Cashore, Visseren-Hamakers, Torres et al. 2016; 
Humphreys, Cashore, Visseren-Hamakers et al. 2017) (see Table 28.4). 

For all these reasons, it is clear that if government officials are to draw on and help nurture 
collaborative certification governance to address public policy problems, they need political 
acumen skills that emerge from training in political science, sociology, anthropology, history, or 
related disciplines that emphasized deep conceptual and theoretical analytical skills on questions 
of power, legitimacy, authority, and political legitimacy. Likewise, policy analytic capacity is 
required to help shape strategic choices capable of understanding why low standard eco-labeling 
competitors emerge and implications for “ratcheting up” standards over time. These skills must 
include an ability to respond in ways that incorporate differences across certification systems 
without alienating most stakeholders. 


Superseding Pathway 


The superseding pathway was identified through political and policy analytic capacities by iden- 
tifying the role of collaborative certification in creating a “learning laboratory” in the hopes that, 
through diffusion and isomorphism, governments and other actors may decide to adopt these 
practices. Unlike the direct pathway, this indirect one requires significant attention and support 
from government officials since they will ultimately be developing the (diffused) regulations. 
Scholars and practitioners have envisioned this role as potentially the most effective for such 
NSMD systems as LEED Green Building certification, which tend to obtain direct support 
from a narrow set of institutional building owners rather than everyday homeowners. Hence, 
the process for broader impact is the diffusion of these standards into municipal building codes 
and related regulations (see Table 28.5). 
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Table 28.5 Superseding = Creating a Learning Laboratory 


Causal Influence Logic Strategic Implications 
Governments adopt NSMD Only attract top producers to trigger diffusion to government 
standards for mimetic, regulations 


normative, or learning processes Certification standards should be maintained at the highest of levels 


The strategic implications for this “superseding” pathway that aims to gradually inform pol- 
icy are different from the direct approach. In particular, strategists should attempt to generate 
and maintain “high standards” because it is through diffusion to government policies that the 
biggest impact can be achieved. Unlike direct pathways, decisions to lower standards to gain 
increased membership are misplaced — because it is learning about best practices, rather than 
generating wide spread support, that leads to indirect influence through impacts on public poli- 
cies. In addition, officials from key government regulatory agencies play a key role and must 
constantly maintain open channels of communication with officials from certification programs 
while maintaining institutional autonomy. 

Key individual policy capacity requirement dynamics include political acumen skills capable 
of understanding and promoting policy diffusion from private governance institutions to tra- 
ditional governmental processes. This requires an ability to understand existing norms within 
policy subsystem dynamics, but also how, by engaging governmental arenas through policy dif- 
fusion processes, norms and discourses might change over time (Dobbin, Simmons and Garrett 
2007). Likewise, policy analytic capacity is key, as this requires understanding and managing 
diverse stakeholders on the one hand, as well as the nuances of public policy process on the 
other hand. Policy analytic skills must also include an ability to master nuances of a specific and 
complicated policy field that incorporates knowledge from a range of social, biological, and 
physical sciences. 


Symbiotic Pathway 


A third pathway captures cases in which NSMD systems emerge to address a gap in international 
agreements or public policies (see Table 28.6). For example, when environmental and social 
activists were concerned that the Kyoto Protocol’s “Clean Development Mechanism” (CDM) 
downplayed non-carbon concerns, such as equity and biodiversity, they did not want to “open 
up” hard-won deliberations for fear they might reduce, rather than increasing, existing protec- 


” 


tions. As a result, some activists turned and championed the CDM “gold standard” certification 
program as a way to fill gaps, encouraging those engaging in CDM activities (Levin 2009). In 
these cases, government officials must have the capacity to recognize that public regulation 
and the NSMD certification system are symbiotic, each reinforcing the other. This means that 
government officials can, quite innovatively, draw on certification systems to fill gaps in public 
policy, rather than seeking changes in their own hard-fought efforts, which could cause duplica- 
tion and undermine support. 

To do this well, policy analytical skills are also required, including intensive training in 
international relations and global governance, especially those bodies of research that are 
targeted towards understanding why states cooperate and implications for compliance and 
effectiveness. These skills are usually taught in departments of international relations and 
global affairs. 
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Table 28.6 Symbiotic Pathway for Addressing Gaps in International Agreements 


Causal Influence Logic Strategic Implications 


NSMD system fills gap in public policy | Government officials can draw on NSMD standards to fill gaps 
approach, reinforcing legitimacy of in existing public policy instruments 
each other Government and certification officials should avoid duplication 
with the other 


Hybrid Pathway 


A fourth pathway that emerges from an application of political acumen and policy analytic 
capacities fits under the broad category of “hybrid” governance (see Table 28.7). This cat- 
egory most closely conforms to the collaborative governance discussion in Kekez, Howlett, and 
Ramesh’s introductory chapter in that it also focuses, ultimately, on how certification might 
help foster changes in both private and public policy by emphasizing policy mixes that might 
provide innovative solutions. These mixes, and the different levels of policy interventions, have 
been disentangled by Cashore and Howlett (2004; Howlett 2009), building on Hall (1993) and 
Sabatier and Jenkins-Smith (1993) and others who, advancing policy analytic capacity, identify 
six different policy cells that officials might draw on to nurture influence. The ability to identify 
whether private or public policy is best for each cell and the potential of developing mixes across 
the cells for shaping impacts over time require the integration of political acumen and policy 
analytic capacity skills. 

To be sure, application of these skills to advance certification through hybrid interactions with 
public policy can yield literally thousands of possibilities, each with its own “causal influence 
logic” for addressing public policy problems. This means that the only way to identify specific 
skills that would be required of individuals is to “work backward” from a specific innovation. 
To illustrate the potential of hybrid governance and the priority of political acumen and policy 
analytic capacity, we reflect on our work on the case of legal verification of global value chains 
(Cashore, Leipold, Cerutti et al. 2016; Cashore and Stone 2014; Cashore and Stone 2012), to 
which so many policy actors and scholars have devoted so much attention in the last ten years. 


Conceptualizing Legality Verification (LV) 


The idea of verifying whether production and manufacturing of consumer goods have fol- 
lowed the laws of the countries of origin has gained increasing interest from those championing 
improved apparel manufacturing facilities and extractive processes within mining, fisheries, and 
oil and gas production. Within forestry, such an interest has coalesced by those seeking to com- 
bat “illegal logging”: i.e., failure to comply with laws and regulations developed by government 
officials such as illegal conversion of forests to other agriculture crops (deforestation) and logging 
that ignores environmental and social regulations. 

Like NSMD systems, LV’s mechanism for weeding out illegal logging is to track legal wood 
along global supply chains that cut across multiple jurisdictions. Likewise, third-party auditing, 
rather than relying on traditional enforcement agencies, is emerging as a key mechanism to 
assure legal compliance. However, unlike certification, these efforts seek to improve government 
capacity to enforce their own laws (Hyde 2009; see Table 28.8). 

For these reasons, it is imperative that government officials have the political and analytic skills 
to distinguish LV from certification since the former can help governments through collaborative 
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Table 28.7 Hybrid Pathway for Reinforcing Compliance With Government Standards 


Causal Influence Logic Strategic Implications 

Influence depends on the specific hybrid Government officials must draw on transnational LV to 
in question. NSMD “look-alikes” have “weed out the worst”, rather than rewarding the top 
emerged that reinforce government Government officials must first focus on collaborative 
policies, rather than private standards governance around supply chain tracking 


Government officials must identify a plan for Phase II 
in order to maintain multi-stakeholder collaborative 
governance beyond business interests 


Table 28.8 Key Features of Legality Verification as TBG 


Role of Governments Sovereignty reinforced, not challenged 
Policy Scope Relatively modest 

Assurance Third-party verification 

Role of Markets Tracking along product supply chains 


Source: Cashore and Stone (2012), drawing on Cashore, Auld and 
Newsom (2002); Cashore, Auld and Newsom (2004a) and Bernstein 
and Cashore (2007b) 


governance in ways that reward, rather than constrain, legal producers. It is precisely for these 
reasons that so many government officials in developing and developed countries are supporting 
the removal of illegal products along global value chains to improve domestic governing capacity. 


Theorizing LV’s Emergence and Evolution: Strategic Implications 


Just how to research and assess a dynamic policy instrument such as LV that operates across mul- 
tiple levels of governance requires applying both political acumen and policy analytic capacity 
such that theoretical and empirical analysis can be integrated to uncover plausible “causal influ- 
ence logics”. Drawing on a range of empirical case studies in Brazil, the United States, the Euro- 
pean Union, China, Malaysia, Cameroon, and Indonesia and theoretical work reviewed earlier, 
scholars applying political acumen and policy analytic skills have conceptualized two phases, 
with distinct “causal influence logics” through which policy actors might nurture and entrench 
LV (Bernstein and Cashore 2007b; Cashore 2012, 2014; Cashore, Auld and Newsom 2002). 
During Phase I, support occurs only when a range of government, business, and environ- 
mental groups all come to recognize some type of strategic organizational interest in supporting 
efforts to “weed out” illegal logging from supply chains. Drawing on policy acumen capacity, 
they theorize that law-abiding businesses will support LV as a policy instrument when they 
evaluate current or potential economic rents associated with weeding out illegal supply (which 
should drive up prices, at least in the short term) and market access as being higher than the 
costs of support (that they incur through standards, auditing processes, and supply chain track- 
ing of legal wood). Hence, policy analytic capacity is fundamental for identifying and building 
supply chain tracking systems capable of weeding out illegal wood as fundament for Phase I, 
requiring that government officials focus attention on developing stakeholder engagement that 
maintains coalitions of businesses and environmental groups. They theorize that a second phase 
of LV, with different logics, allows for regulatory expansion once LV of the entire supply chain 
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is institutionalized such that shirking among consumers and/or producers is considered norma- 
tively unacceptable. In this case, government officials must have the conceptual skills to identify 
Phase II, since it can only be measured after their efforts are unleashed, and the managerial skills/ 
training from economics to know that when such conditions exist, increases in regulations will 
impose costs on consumers, rather than firms (Cashore and Stone 2012). 

Application of a high degree of political acumen and policy analytic capacity among govern- 
ment officials in China, the US, and Indonesia helps explain why there was a flip from lukewarm 
or nonexistent support in the early 2000s to strong support by the late 2000s (Cashore and 
Stone 2014). Likewise, these skills allow policy actors to disentangle the type of policy problems 
LV could address, such as promoting responsible extractive practices, from more thorny policy 
problems, such as conserving biodiversity over extractive interests. 


Direct and Indirect Evolutionary Pathways: Managerial 
Capacity Requirements 


Given our review has identified critical political acumen and policy analytic capacity that cut 
across and help the identification of four distinct causal pathways, what then are the managerial 
requirements of policy actors who seek to influence them? 

We can identify skills, drawing on Howlett and Ramesh (2016), which, like political acumen 
and policy analytic capacities, cut across all four pathways: excellent communication, budgeting, 
leadership, and team-building skills and the ability to conduct a range of cost-benefit analysis 
calculations, from optimization efforts to multi-goal policy analysis. These skills are taught in 
public administration and public policy schools. In addition, as NSMD systems follow business 
principles as much as government bureaucracies, managers ought to be aware of and trained in 
management techniques discovered and applied by key business management scholars such as 
Peter Drucker (Drucker 2001), who teaches how a “knowledge worker” leader can generate 
learning by empowering employees to realize organizational objectives through decentralized 
arrangements and collaborative learning (Senge 1990). Mintzberg and McHugh’s (Mintzberg 
and McHugh 1985) findings about the way in which leaders can foster organizational coherence 
allows leadership “at the top” to foster innovation across all levels of the organization (Bushe and 
Marshak 2016) and the types of organizational forms that might nurture adaptive management 
(Laloux 2014). 

These are clearly cross-cutting managerial capacities that any policy actor should master 
to effectively engage with colleagues and employees within the NSMD system and relevant 
government and intergovernmental arenas. However, whereas these skills are agnostic about 
particular organizational goals, NSMD certification systems were designed to address particular 
problems. This means that managerial skills must always be adjudicated for their ability to address 
these goals, rather than, as can inadvertently occur, maintaining the existence or growth of the 
system itself becoming the goal. When managerial and employee identities become focused on 
organizational reproduction rather than problem solving — such as direct and hybrid pathways 
being stuck in Phase I — the entire NSMD project is threatened, which will serve to undermine 
legitimacy and authority and stakeholder support. As abilities to make these distinctions are the 
functions of political acumen and policy analytic capacity, it follows that these managerial func- 
tions must always be applied third in the sequence. 

At the same time, we can also inductively identify “Achilles” heel micro-level” management 
techniques that are different according to the pathway in question. For example, the direct path- 
way requires that managers to understand and conduct sophisticated supply chain management 
techniques that are taught in business schools and that take into account a range of technological 


366 


Non-State Market-Driven Tools 


Table 28.9 Dominant “Achilles’ Heel” Individual Managerial Skills for Traveling NSMD Influence 


Pathways 
Pathways Tiaining 
Direct Supply chain management: technological innovations for tracking (GIS, remote 
sensing, data sharing) 
Superseding Legal: knowledge of relevant countries’ legal practices and procedures, public 
policy legacies, and compliance and reporting requirements 
Symbiotic International compliance: knowledge of treaty obligations, standard operating 


procedures in relevant convention 
Hybrid All of the above 
(Legality Verification) 


innovations for creating efficient systems. Conversely, superseding pathways require strong man- 
agerial training in relevant domestic countries’ legal requirements, often taught in law schools, 
through which diffusion must occur while symbiotic requires knowledge of international treaty 
mechanisms and compliance management challenges, usually the domain of schools of global 
affairs and international relations. Given its complexity, it is unsurprising that the only path- 
way that does not have an Achilles’ heel micro-management capacity building challenge is the 
Hybrid LV approach, in which all three micro capacities are key (see Table 28.9). 


Conclusion 


This analysis has focused on the individual-level capacity requirements. We recommend that 
future efforts apply the framework to identifying relevant organizational and system-wide capac- 
ities as well. One finding is clear: emphasis on managerial capacities at the expense of political 
acumen and policy analytic capacities leads not only to a limited ability to anticipate collabora- 
tive governance failures but also to the inability to anticipate fully successful pathways and strate- 
gies for traveling them. 
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Considering the conceptual, methodological and developmental relevance of the matter, the 
evaluation of public policies, programs or processes is pointed to in the contemporary research 
agenda as an essential tool for the improvement of social public policies, at all government levels 
and in a global scenario. Public policy assessments are, in fact, dynamic and complex, given that 
many policies involve multiple actors, levels of government and different perspectives. It is also 
worth mentioning that there is a global effort aimed at building an agenda based on integration 
among all international organizations, guided mainly by the achievement of the 17 Sustainable 
Development Goals (SDGs) of the United Nations Development Program. In this context, 
this chapter is characterized as a proposal for a theoretical framework, aiming to build a com- 
prehensive and integrated analysis of public policy assessments based on the relevant literature 
and, moreover, to understand and propose reflections about the current model regarding the 
relevance of investigative designs and outcomes for each of the types of assessment. International 
trends are highlighted, as well as the variety and complexity of applications, methods, tools, 
approaches, moments and actors involved in the evaluation process. It is concluded that there 
is a great effort involving countries, governments and multilateral agencies to strengthen the 
study of evaluation, looking for best practices and theories, thereby contributing greatly to the 
creation of an evaluative “culture”, capable of encompassing the complexity and specificities of 
each policy or program in question. 


Introduction 


The study of public policies encompasses the understanding of the forms of action taken by the 
state, as well as the coordination of efforts and articulations between other players in the politi- 
cal arena. Noted, especially in modern democracies, is that this type of analysis is gaining ever 
more prominence, which has contributed to the strengthening of these institutions. As a result, 
the search for analytical processes is a way to decrease the complexity of the object and simplify 
the analyses. Accordingly, the division into stages was established under the aegis of the political 
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cycle, as well as in relation to the various phases of the political process arranged in a sequential 
and interdependent manner (Howlett et al. 2013; Valle-Cruz et al. 2020). 

Through its ample diffusion, the political cycle has become the most widely used structure for 
the ordering and systematization of research on public policies. However, there are limitations to 
the adopted scheme, such as the simplification of extremely complex facts, the lack of linearity of 
policies in a real context and the possibility of changes in the phases and order adopted in the polit- 
ical cycle (Valle-Cruz et al. 2020). For this chapter, we chose to describe the stages of the political 
cycle based on Howlett et al. (2013), and as this is a holistic field with several developments, this 
study proposes the specific analysis concerning the structuring of the stage in the political cycle 
denominated as evaluation, especially in a context of the large-scale production of information and 
possibilities that arise for real-time evaluations (Feinstein 2019). Figure 29.1 presents the stages of 
the political cycle in accordance with the resolution applied to the problem. 

In a scenario of scarce resources and recurrent economic crises, the evaluation of public poli- 
cies and programs is desirable and necessary, which contributes greatly to the identification and 
development of efficient mechanisms for dealing with financial, human and structural resources 
(Ala-Harja and Helgason 2014). Through evaluative processes, substantiation of the critical 
factors concerning the success of a given policy can be sought, which encourages debate and 
decision-making by managers and civil society, as well as promoting the constant improvement 
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Source: Elaborated by the authors based on the literature review and on Howlett et al. (2013) 
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of already-developed policies (Silveira et al. 2013). Even with the wide range of definitions for 
the evaluation of actions, there is convergence in relation to the contributions generated: knowl- 
edge for making decisions, organizational learning and improvement to the policy in question 
(Queiroz and Capelari 2020). 

The effective use of evaluative processes can contribute to the transparency of public actions, 
as well as present to the public ways of controlling and following government actions, thus guar- 
anteeing the legitimacy of policies or programs developed through such (Weiss 1999; Ala-Harja 
and Helgason 2014; Mokate 2002; Vedung 2009; Ramos and Schabbach 2012). However, note 
should be made that, due to their political nature, evaluations can contribute to the mainte- 
nance or not of a given policy, which may be linked to the proposals and strategies of the parties 
involved (Ramos and Schabbach 2012; Cruz 2015). 

Highlighted here also is that the challenge of successful evaluations permeates social trans- 
formation in a world where the COVID-19 pandemic has brought to light the economic and 
social chasms between countries, groups and individuals, in addition to the urgency to improve 
the human-nature relationship. Through such, the importance of public policy modeling and its 
limitations are made evident, along with the need for evaluative, adaptive and transformational 
models (Picciotto 2020). To this end, evaluative practices should seek to improve their processes 
and constantly update their range of tools and must seek to change the focus of their actions, 
becoming transformative and dynamic (Feinstein 2019). Furthermore, the availability of infor- 
mation, big data and the use of artificial intelligence provides evaluation opportunities in very 
near real time, in addition to facilitating the incorporation of the evaluation across all phases of 
the political cycle, replacing the notion of the evaluation only at the end of the process (Feinstein 
2019; Picciotto 2020; Valle-Cruz et al. 2020). 

Given the summary of the presented scenario and aiming to contribute toward the debate on 
the evaluation of public policies, this chapter sets out the next topics for the historical review of 
the study on evaluation, in addition to providing details to the existing models, types and meth- 
odologies in the literature and recent international trends included in this discussion. 


Studies on the Evaluation of Public Policy 


The perspective from literature considers the proliferation in evaluative studies as being the end of 
the 1950s. According to Vedung (2010), the historical process of studies on evaluation in the global 
context can be divided into four waves of diffusion. Each wave is indicated for a period of time and 
molded according to social, political and economic questions. The construction of this metaphor 
refers to the constant renovation of the scenario, resulting from the repetitive movement of the 
waves and the constant deposit of sediments to create a base (Vedung 2010). This chapter derives 
from the article published in 2019 (Oliveira and Passador 2019) and advances the understanding 
and practices of assessment in a context of growing economic and social imbalance. 

The first wave or phase of diffusion of evaluation studies dates back to the late 1950s, with 
consolidation occurring in the mid-1960s (Derlien 2001; Trevisan and Bellen 2008; Vedung 
2010; Picciotto 2015) and is characterized as “the science guided wave” (Vedung 2010). The 
studies one finds within this time frame are linked to the emergence of the welfare state, mainly 
in those countries with reformist aspirations during that period[ highlighted among such are the 
United States of America (USA), France, Germany, Sweden and Canada (Derlien 2001; Vedung 
2010; Picciotto 2015). The second period, or second wave, described by Vedung (2010), 
denominated as “the dialog guided wave” was configured during the 1970s, in a context in 
which researchers driven by progressive ideals argued that evaluations should be more pluralistic, 
democratic and dialogical (Vedung 2010; Picciotto 2021). The insertion of all actors involved in 
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the intervention under analysis is discussed, from researchers and politicians to operators of the 
policy or program in question (Vedung 2010). The authors Rossi and Wright (1984) highlight 
the growth in the use of qualitative studies in this period, a model that converged inclusively 
with the thinking of that time, while justifying their use based on lower research costs on a local 
level and greater use and flexibility for managers. Such alternative models would provide bet- 
ter support to the participatory scope, as well as closer contact with reality (Rossi and Wright 
1984; Vedung 2010). Synthesizing the characteristics of this wave, Vedung (2010, 270) explains 
that “in contrast to the wave induced through science, the wave guided through dialog rested 
on communicative rationality. Instead of producing truths, dialogic evaluation would generate 
broad agreements, consensus, political acceptability and democratic legitimacy”. 

On the other hand, the third wave, denominated by Vedung (2010) as the “neoliberal wave” 
and introduced at the beginning of the 1980s, modified once more the standard of evaluation 
studies. In their studies, the authors Vedung (2010) and Picciotto (2015) describe this period as 
a neoliberal inundation into the field of evaluative studies, with such focusing on a reduction of 
the state while promoting at the same time the free market and public-private partnerships. The 
emerging management model was the New Public Management (NPM), with greater focus on 
results in regards to processes. The study in Vedung (2010) complements NPM by arguing that 
customer orientation, decentralization, privatization and deregulation have become the new 
slogans of evaluation processes. Contributing further, Ala-Harja and Helgason (2014) indicate 
that, in this phase, the evaluation was understood as a tool able to justify policies and reallocate 
financial resources. In conclusion, Vedung (2010) synthesizes that the evaluation activities of 
the third wave, which were mainly focused on accountability, performance measurement and 
consumption, seek quality of services provided and comparative evaluations. 

Continuing with this theme, the fourth and last wave described by Vedung (2010) is denomi- 
nated as “the evidence wave: the return to experimentation”. This wave of alternative studies 
took shape in the 1990s and 2000s, mainly in North Atlantic and Nordic countries. These are 
evidence-based studies, so Picciotto (2015) emphasizes that those involved in evaluation studies 
are “surfing” this fourth wave. Studies focused on scientific evidence aim to separate that which 
works from that which does not (Vedung 2010; Picciotto 2015). The author highlights that stud- 
ies with experimental approaches that use randomization or quasi-experimental research have 
as their main objective the verification of the impacts caused by policies and public programs 
(Vedung 2010; Picciotto 2015). The study (Vedung 2010) classifies this movement of incessant 
search for scientific proof as a return to the science-based model, but under a new light. 

Nevertheless, in a context of pandemic and global economic and social crises, there are signs 
of the emergence of a new wave of assessment or fifth wave. New opportunities were generated 
by the availability of information, digitization of processes and scarcity of human and financial 
resources, as well as the collaborative relationships between countries for the development of 
new practices and technologies, especially in the health sector. Adhering to social trends, this 
new wave presents itself with evidence-based assessment characteristics, but with a greater focus 
on the socio-environmental impact and on the growing participation of society. 


Typologies and Approaches for Public Policy Evaluations 


Starting out with the assumption that evaluations are not equal, Cohen and Franco (2008) 
and Ramos and Schabbach (2012) suggest the description of the different forms of evaluation 
according to a series of criteria, such as use, time, who performs it, to whom it is destined and 
the desired outcome, among others. According to Faria (2005), one can point to at least four 
types of use for the evaluation, those being instrumental, conceptual, persuasive and clarification. 
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Instrumental use refers to the importance of quality and proper dissemination of obtained 
results, in order that the evaluation becomes something that is tangible and of feasible imple- 
mentation. The study by Weiss (1998) adds that instrumental use for evaluations can guide 
decision-makers as it allows for a return to the history of the analyzed program and provides 
feedback to assessment practitioners as well as highlighting the objectives established for the 
program. The second use, denominated as conceptual, is frequently applied to those that do 
not have direct contact with the formulation of the program (local technicians); it is, therefore, 
about providing information on the policy or program, presenting as such its nature and opera- 
tion, as well as introducing it to the impacts and results achieved (Faria 2005). Following on, 
the third use, called persuasive, is described by Faria (2005, 103): “[T]his occurs when used to 
mobilize support for the position that decision-makers already have on necessary changes in 
policy or program”. The evaluation, at this moment, has the function of legitimizing the desired 
changes and winning over new supporters to this movement. Finally, the use of clarification is 
proposed as a result of the accumulation of knowledge arising from several evaluations that have 
already been carried out (Faria 2005) and has the capacity to guide the government agenda. 

Regarding the moments of the evaluation, one can consider the ex-ante evaluation and ex- 
post evaluation. Ex-ante evaluations are those carried out before decision-making, and one can 
say that it has as its goal to aid in decision-making, in terms of the implementation of policies 
or social programs (Cotta 2001; Cohen and Franco 2008). Furthermore, Ramos and Schabbach 
(2012) affirm that, in this type of evaluation, a diagnostic is generated on the observed situa- 
tion, thus contributing to the suitability of the available resources to the proposed objectives 
(feasibility check), additionally being able to serve as a guiding tool for the maintenance and/or 
formulation of policies or programs. However, the classification ex post indicates that the evalu- 
ations are performed during the execution of a project or at its end. When the evaluation refers 
to the intermediary moment (process evaluation) denominated by Scriven (1991) as formative 
evaluations, the conclusions may base decisions on continuity or discontinuity of a given policy 
or program (Cotta 2001; Cohen and Franco 2008; Trevisan and Bellen 2008; Ramos and Schab- 
bach 2012). However, in regards to terminal or summative evaluations (which measure impact 
and result, for example), the ex-post evaluations provide information concerning the experience 
achieved and can support future decisions on the same policy or program or even contribute to 
the construction of an evaluative model for another objective or similar experiences (Scriven 
1991; Cotta 2001; Cohen and Franco 2008; Trevisan and Van Bellen 2008; Ramos and Schab- 
bach 2012). Based on the literature, the inference can be reached that ex-post evaluations are 
more widespread both in academia and practical applications and so present more founded and 
sophisticated methodologies. Regarding the classification of the evaluation from the perspective 
of those who carry out or participate in it, it can be said that there is a consensus in the literature 
with regard to the four distinct types: external evaluation, internal evaluation, mixed evaluation 
and participatory evaluation. Table 29.1 illustrates these differences. 

External evaluations add exemption to the evaluation practice; on the other hand, however, 
the external evaluator has greater difficulty accessing data. In addition, one can mention as an 
advantage the greater objectivity of external evaluators, as these do not participate in internal 
processes, as well as the possibility of comparison with similar programs (Cotta 1998; Cohen 
and Franco 2008; Ramos and Schabbach 2012). With respect to internal evaluations, the advan- 
tages presented in the literature deal with the decrease in conflicts generated by the insertion of 
external people or organizations, the possibility of revision and learning in terms of the processes 
executed in a particular policy or program and greater theoretical and technical knowledge 
of the evaluated specificities. However, internal evaluations risk losing objectivity and provid- 
ing only partial conclusions as these are developed by those involved in the formulation and 
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Table 29.1 Types of Evaluation According to Who Performs/Participates 


Type of Evaluation Producer Organizer User 

External — performed by agents External External/internal External/ 
outside the organization internal 

Internal — performed in the Internal External/internal External/ 
organization managing the internal 
policy or program 

Mixed evaluation — composed of External and internal External and/or External/ 
external and internal agents, internal internal 


with the objective to keep 
advantages from both evaluations 


Participatory evaluation — the Evaluation board Evaluation board External/ 
main objective is to reduce the (agents who are (agents who are internal 
distance between evaluators and part of the policy or part of the policy or 
beneficiaries program, external program, external 

evaluator members evaluator members 
and beneficiaries) and beneficiaries) 


Source: Elaborated by the authors based on Weiss (1998), Cohen and Franco (2008), Vedung (2009), 
Ramos and Schabbach (2012) and Oliveira (2018) 


implementation of the analyzed program (Weiss 1998; Cotta 1998; Cohen and Franco 2008; 
Vedung 2009; Ramos and Schabbach 2012). 

Mixed evaluations aim to join efforts and ensure synergy between the actors involved in the 
action. The potential of all of the models is then combined, while reducing the impacts or biases 
presented individually (Cohen and Franco 2008; Ramos and Schabbach 2012). Finally, the partici- 
pative evaluation is presented, which holds as its main objective to decrease the distance that exists 
between evaluation organizers and producers and the beneficiaries of the evaluated programs. In 
this model, the participation of users in all phases of the political cycle is allowed, favoring opinions 
and meeting the specific demands of these actors (Cohen and Franco 2008; Ramos and Schabbach 
2012). For Patton (1999), the participative evaluation should consider the priorities and effects 
while establishing the common definitions for the evaluators as well as the beneficiaries. In Spink 
(2001), the importance of the participative evaluation is shown and, as already highlighted, has the 
capacity to strengthen ties between those involved, thus collaborating with the maintenance of the 
practice of democratic evaluation. However, the dimension of the policy or program evaluated 
must be taken into account; in this regard, Ramos and Schabbach (2012) suggest that participative 
evaluations are adequate only in small projects or local programs. The study by Ceneviva and Farah 
(2012) highlights that bureaucrats should not work in an isolated fashion; contrarily, evaluation is 
a tool of transparency allied to public acts, where citizens can and should inform themselves and 
inspect the actions put forward by political players. 

Evaluations can also be classified as qualitative, quantitative and mixed methods. Synthetically, 
Weiss (1998) classifies qualitative research as that which is presented in words and quantitative in 
numbers. The author complements this by affirming that quantitative evaluations collect data and 
transform it into numerical information, which is strongly supported by statistics as well as math- 
ematical and econometric methods. Quantitative approaches can use sophisticated tools in data 
analysis that identify relationships between variables and can encompass a large sample universe in 
addition to providing information on the extension and distribution of a particular phenomenon 
(Weiss 1998). On the other hand, qualitative evaluations tend to use less structured tools, such as 
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interviews, observational techniques and document analysis (Weiss 1998). The authors Rossi and 
Wright (1984) assert that for social research, as in the case of public policy studies, the qualitative 
approach presents advantages as it has the capacity to generate approximation with reality and is 
flexible for contemplating the complexity involved in such policies. 

The difference between approaches generates opportunities for contemplation, the joining 
of similarities and increasing synergy between the different parts (Mokate 2002). It is in this area 
that the mixed methods approach finds its niche, which is nothing more than the combination 
of qualitative and quantitative assessments. Mokate (2002, 114) argues that “in particular, the 
evaluation can be applied to quantitative and qualitative methods, as well as for the generation of 
different types of information which, together, correspond to the different questions proposed 
by the evaluation process”. Research by Bloch et al. (2014) upholds that the use of mixed meth- 
ods collaborates with the employment of the best data sources available, with the intent of pro- 
viding comprehensive and robust answers. Nevertheless, the authors present as limitations to the 
use of mixed methods the time, resources and commitment to build the pragmatic aspects of the 
evaluation (Bloch et al. 2014). Following suit, Costa et al. (2013) add that quantitative research 
leads to the generalization of results shaped through standardized information. In contrast, quali- 
tative research will likely provide data relating to circumstances and social environments, giving 
emphasis to cultural and contextual dimensions. 

On the topic of the methodology applied to the evaluation, Costa and Castanhar (2003) 
group the evaluations into process evaluation, goal-based evaluation and impact evaluation. 
Evaluation, once considered the way to measure the performance of policies or programs, with 
such performance being the reaching of goals (Costa and Castanhar 2003) or the assessment of 
results (Ramos and Schabbach 2012), needs criteria to structure itself and, finally, to measure 
the consequence of that contracted (Costa and Castanhar 2003). Table 29.2 presents the types of 
evaluation according to the problem that the evaluation answers, as well as the benefits of its use. 

Here, three basic dimensions prevail for measuring the obtained results (Costa and Castanhar 
2003; Sano and Montenegro Filho 2013): 


Efficiency: from economic origins, with the meaning to reach the program objectives, 
while prioritizing the standards established, with the lowest cost-benefit relationship pos- 
sible. Efficiency is related to the process stage in an evaluation flowchart. 

Efficacy: measures the degree to which the goals and objectives were achieved, translating 
in a simplified mode the obtained result. 

Effectiveness: also dealt with in the literature as the measure of impact, it indicates the 
positive effects in relation to the target audience of the program. It is a wider dimen- 
sion, as it analyzes the economic, sociocultural, environmental and institutional aspects: 
that is, effectiveness measures the quantity as much as the quality of the goals achieved 
by the program. Effectiveness is related to the transformation stage in an evaluation 
flowchart. 


Noteworthy here also is that the importance of the investigative designs and outcomes for each 
one of the evaluation types — process, goals/results and impacts — according to the problem that 
it deals with. In light of this necessity, Table 29.3 describes, without claiming to have closed 
the debate on the topic, each of the designs and outcomes presented in the literature (Scriven 
1991; Weiss 1998; Cotta 1998; OECD 2002; Costa and Castanhar 2003; Vedung 2009; Ramos 
and Schabbach 2012; Silveira et al. 2013). Therefore, the connection between the evaluative 
approaches and the methodology applied to the evaluation is imperative with regard to the 
design and outcome used; such variables are also recorded on Table 29.3. 
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Table 29.2 Types of Evaluation According to the Problem to Which the Evaluation Is Responding 


Type Description Moment of Evaluation Benefits of Use 
Process evaluation Objective is to identify Ex-ante The adequate use of 
problems or obstacles Formative information collected in 


in the implementation 
of processes in a 
systematized way. Its use 
allows for contemplating 
the evolution between 
planning and execution. 
Evaluation of It is the most traditional Ex-post 
goals or results model among evaluation Summative 
methodologies. It 
consists of measuring the 
success of the program or 
project against previously 


established goals. 
Impact evaluation It covers not only the Ex-post 
results achieved in Summative 


terms of efficiency, 
effectiveness and efficacy 
but also the changes that 
occurred in the target 
population as a result of 
the implementation of 
the policy or program 
evaluated. 


the evaluation process 
permits small adjustments 
or change of direction 

to be made during 
implementation and 
execution of the policy. 


The effects and 


consequences of a given 
policy are measured, 
determining its success 
or failure based on 
effective changes in the 
recipient populations. 


This evaluative model 


uses methodological 
frameworks that provide 
cause-and-effect 
relationships between the 
actions performed by the 
program and the effects 
promoted in the target 
community in a broader 


sense. 


Source: Based on Scriven 1991, Costa and Castanhar 2003, Ramos and Schabbach 2012 and Oliveira 2018 


Table 29.3 Connections Between Designs, Evaluative outcomes, Approaches and Types of Evaluation 


Design/Outcome Short Description 


Approach and Type of 
Evaluation 


Cost-benefit The analysis is designed to assess the monetary benefits of 
an intervention, reporting both costs and benefits, in 
order to compare alternatives and choose the best cost- 
benefit ratio. 

Cost- Comparison between alternative forms of action to select 

effectiveness the program/project that meets the objectives with 
the lowest cost. In addition to supporting the cost 
evaluation, it also considers the consequences of one 
particular intervention being completed over another; 
it is commonly used due to it possessing both monetary 
and non-monetary benefits as the outcome unit. 


Cost-utility Considered a derivative of the cost-effectiveness analysis. It 
compares not only costs and benefits but also the perception 
of life quality or social impact of the target population. 


Quantitative approach 
Process evaluation 
Evaluation of goal/results 


Quantitative approach 
Process evaluation 
Evaluation of goal/results 


Mixed methods approach 
Impact evaluation 
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Design/Outcome Short Description Approach and Type of 
Evaluation 
Experimental These point toward the formation of two populations: the Quantitative approach 
studies group with the program/project (treatment) and another Impact evaluation 
group without the program/project (control). 
The objective is to verify the changes generated by the 
program/project. 
Quasi- Similar to the experimental model; however, the groups are Mixed methods approach 
experimental not randomly selected; it is based on a choice between Impact evaluation 
studies those that belong to the target population and fit into 
one of the groups (treatment or control). The objective 
is also to check for changes that have occurred in these 
populations. 
Documentary Method of data collection involving the use of information Quantitative approach 
methods for other purposes, such as statistics or data produced by Process evaluation 
the agency itself and/or other written documents. Evaluation of goal/results 
Interrogative Strategy to accumulate data through questioning people, Quantitative approach 
methods appropriated from interviews, questionnaires and the Process evaluation 
like. Evaluation of goal/results 
Impact evaluation 
Observational Data collection strategy in which the investigator, through Quantitative approach 
methods a site visit, makes on-site observations. In direct Process evaluation 


Meta-evaluation 


Triangulation 


observation, participation in the process of evaluation is a 
pretext to allow for the collection of impartial data. 

The term is used for designed evaluations that aggregate 
results from a series of evaluations. It can also be used to 
denote the assessment of an evaluation to assess its quality 
and/or to assess the performance of assessors. 

It is the use of several different measurements or measuring 
instruments to obtain information about the same action, 
condition or thing. 


Evaluation of goal/results 
Impact evaluation 
Mixed methods approach 
Evaluation of goal/results 
Impact evaluation 


Mixed methods approach 
Process evaluation 
Evaluation of goal/results 
Impact evaluation 


Source: Elaborated by the authors 


In conclusion, confronted with the variety and complexity of the applications, methods, 
tools, approaches, moments and players involved in the evaluation process, it is necessary to 
synthesize the information in order to illustrate the main divisions presented in this study, based 
on the consulted literature, without the intention of relating them in a temporal or hierarchical 
way, as shown in Figure 29.2. 

The next section explores the international tendencies surrounding the evolution of evaluation 
studies in order to relate these through the point of view of types, methodologies and outcomes. The 
strengthening of evaluation practices in a context of economic, social and health crises at a global level 
is also highlighted; this points to a new wave of evaluation studies that we are calling the fifth wave. 


Recent National and International Tendencies 


In light of scenarios becoming ever more complex and turbulent, global tendencies converge 
toward the structuring and strengthening of actions in international evaluation, which are 
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Figure 29.2 Conceptual Map of Public Policy Evaluation 


Source: Elaborated by the authors 


supported by government organs, independent agencies, multilateral agencies and civil society 
organizations. It is clear that evaluative activities are important for good governance, while also 
being able to contribute to the advance of public policies that are of a more effective, effica- 
cious and efficient (Ramos and Schabbach 2012; EvalPartners 2015). As a result, this topic maps 
developed actions and those present actions carried out by international organizations while also 
leaving the topic open to further discussion. 

The United Nations (UN), the World Bank, the Inter-American Development Bank (IDB), 
the Organization for Economic Cooperation and Development (OECD), the Economic Com- 
mission for Latin America and the Caribbean (ECLAC) and the Latin American Center for 
Development Administration (CLAD) emerge as protagonists in the generation of evaluative 
methodologies and projects (Ramos and Schabbach 2012). Noteworthy here also are actions 
from other agents. Picciotto (2019) points out that at the turn of the century, there were only 20 
evaluation associations; however, since then the number has grown considerably. EvalPartners, 
formed by the International Organization for Cooperation in Evaluation (IOCE), has identified 
a total of 158 associations or networks, of which 135 are at the national level. 

In this context, 2015 was established by EvalPartners and approved by the UN as the Inter- 
national Year of Evaluation. The organization, constructed on a partnership between the Interna- 
tional Organization for Cooperation in Evaluation (IOCE) and the United Nations Children’s 
Fund (UNICEF), sought partnership with various important organizations, with the aim of creat- 
ing a worldwide network of volunteer evaluators who can contribute to public policy evaluations 
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in their countries (EvalPartners 2016). According to the guidelines of the organization, the incen- 
tive aims to contribute toward improving the capacities of civil society organizations (CSO), as 
well as involving these in a strategic and expressive fashion in the processes of national evaluation, 
with the intent of developing systems of evaluation while guaranteeing equity in the resulting 
actions (EvalPartners 2016). To this end, they developed the Global Assessment Agenda (Eval- 
Partners 2016). The document, approved by the delegates of the Global Evaluation Forum held 
in Nepal in 2015, enables a roadmap of actions for the future of evaluation and the strengthening 
of evaluators from different countries and organizations (EvalPartners 2016). 

The European Union also developed a project for evaluating structural policies of the bloc, 
along with the identification and use of innovative practices of evaluation methodologies, in 
addition to multisector involvement, with the intent to reduce poverty (Sophie Project 2012). 

In the Latin American context, the Latin American and Caribbean Network of Monitoring, 
Evaluation and Systematization (ReLAC), in conjunction with Evaluation Capacity Develop- 
ment in Latin America (FOCEVAL), the Costa Rican Ministry of National Planning and Eco- 
nomic Policy (MIDEPLAN) and the German Institute for Development Evaluation (DEval) 
conducted a series of public consultations with experts and evaluation professionals during 2014 
and 2015 and presented the first drafts at the IV ReLAC Conference in Peru in 2015. In August 
2016, they released a document with a summary of this proposal under the title “Evaluation 
Guidelines for Latin America and the Caribbean”, translated to Portuguese in 2019, with the 
objective to enrich the common framework of guidance for achieving high-quality evaluations, 
promoting the theoretical and practical training of assessors in the region and instigating the 
culture of evaluation on the continent (Bilella et al. 2016). 

Noteworthy still is that there exists a global effort aimed at building an agenda based on 
integration among all international organizations, guided mainly by the achievement of the 17 
Sustainable Development Goals (SDGs) of the United Nations Development Program (PNUD 
2016). The first five years projected by EvalAgenda2020 have as their priority the creation and 
the strengthening of evaluation mechanisms of SDG. Table 29.4 summarizes the main actions 
(PNUD 2016; ONU 2016; Evalpartners 2015; OECD 2016; Sophie Project 2015; Bilella et al. 
2016; IBGE 2018; FGV CLEAR 2021; CIDACS 2021). 


Table 29.4 Structuring Actions of Indicators and Evaluation Developed by National and International 


Organizations 

Organization Developed Action Brief Description 

ONU (2015) “Transforming Our Document that established the 17 Objectives and 
World: Agenda the 169 Goals of Sustainable Development at 
2030 for Sustainable the UN General Assembly in 2015. 
Development” 

ONU (2016) First SDG follow-up Inaugural report that gives an overview of the 17 
report SDGs, using available data to highlight some 

critical points and challenges. 
ONU (PNUD Follow-up on Agenda Document with initial subsidies for the 
2016) 2030 identification of national indicators related 


to the SDGs, which hold the objective of 
presenting the available indices for monitoring 
and evaluation of the fulfillment of the 
established goals. 


(Continued) 
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Table 29.4 (Continued) 


Organization Developed Action 


Brief Description 


SDG at the 
Brazilian 


Digital platform SDG 


Institute of 
Geography 
and Statistics 
(IBGE) 


EvalPartners 
(2015) 


EvalAgenda2020 


OECD (2016) “Measuring Distance 
to the SDGs 
Targets: A Pilot 
Assessment of 
Where OECD 
Countries Stand” 

SOPHIE “Evaluating the 
PROJECT Impact of Structural 
(2015) Policies on Health 

Inequalities and 
Their Social 
Determinants, and 
Fostering Change 
(SOPHIE)” 


ReLAC “Evaluation 
Guidelines for Latin 
America and the 
Caribbean” 


Developed by teams from the Informatics 


Directorate of the IBGE, it provides an 
information system for following up the 2030 
Agenda in Brazil. It constitutes a first set of 
global indicators, elaborated collaboratively 
between global institutions, with the purpose 
of presenting regularly produced national data, 
methodology and internationally established 
standards. 


The Global Evaluation Agenda (GEA) 2016— 


2020, which is the first long-term global 

view for evaluation, establishes priorities and 
offers training and forums, with the objective 
of creating an environment conducive to 
evaluations and one that will reach the 

goals established in the 2030 Agenda. The 
collaborative construction involves interested 
parties, including governments, parliaments and 
civil and academic society. 


Pilot study of the starting positions, in relation to 


the SDGs, of some OECD countries. It aims to 
help governments develop national action plans. 
It uses the wealth of statistical data obtained 

by OECD members, harmonized through the 
tools and processes found in the organization. 


Health inequalities are strongly affected by the 


circumstances in which people are born, grow, 
live, work and age, as well as the policies that 
influence those circumstances. The SOPHIE 
Project has as its objective to generate new 
evidence on the impact of structural policies 
on the inequalities in health and develop 
innovative methodologies for the evaluation 
of these policies in Europe. It is coordinated 
by the Barcelona Public Health Agency and 
financed by the European Union Research 
Fund. 


Instrument to foster the culture of evaluation 


in public and private institutions, guiding the 
practice and training of evaluators, ensuring 
credibility, transparency and the integral 
character of the evaluation (Bilella et al. 
2016). The guidelines were divided into five 
dimensions of analysis: (1) rigorous 


evaluation, (2) adequate evaluability, (3) evaluation 


conducted in accordance with ethical and legal 
principles, (4) adequate cultural understanding 
and (5) relevance and usefulness. 
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Organization Developed Action Brief Description 


FGV Clear Development of FGV EESP Clear is one of the six regional 
various projects in centers that make up the CLEAR Initiative, a 
conjunction with global action of various entities and countries 
government agents, that aims to improve policies and programs 
private institutions, through the strengthening of capacities and 
civil society and monitoring and evaluation systems. The 
academia projects are aimed at formulating and/or 

improving the management of public policies 
and social programs based on results in four 
areas of action: (1) diagnosis; (2) monitoring: 
monitoring systems, building indicators; 

(3) evaluation: logical framework, process 
evaluation, cost-benefit analysis, impact 
assessment; and (4) institutional capacity 
building. 

Cidacs — The Development of These studies aim to assist and support decision- 


Oswaldo Cruz studies with 


Foundation of large volumes 

Bahia (Fiocruz of information, 

Bahia) integrating national 
databases and 
providing products 
(databases) for 


research 


making in public policies for the benefit 

of society. Toward this end, Cidacs has 
multidisciplinary teams trained in different 
areas, such as epidemiology, collective health, 
statistics, economics, bioinformatics and 
computing. Cidacs works with the following 
research platforms: Cohort of 100 Million 
Brazilians; Zika Platform; Technologies and 


Innovations for the Unified Health System 
(SUS); Equity and Urban Sustainability; 
Bioinformatics and Genetic Epidemiology 
(Epigen) (CIDACS 2021). 


Source: Elaborated by the authors 


There is a clear understanding that evaluation activities are important for good governance 
and can contribute to the advancement of more effective, efficacious and efficient public policies 
(Ramos and Schabbach 2012; EvalPartners 2015). One notes that the deduction can be made 
that in academia as in government research organizations, when it comes to the strengthening 
of the study of evaluation, these seek best practices and theories, which contribute greatly to the 
creation of an evaluative “culture” capable of embracing complexity and the specifics of each 
policy or program in question (PNUD 2016). 


Final Considerations 


In line with the search for articulated models for the development of government strategies to 
fight poverty and promote citizenship, several international organizations have been develop- 
ing integrative and intersectoral proposals, with the aim of improving the quality of life of the 
world population. Actions such as the establishment of the UNDP SDGs and comparative 
and shared studies among OECD countries are examples of such efforts (Oliveira and Passador 
2019). It also highlights clues for the emergence of a new wave (fifth wave) that will accom- 
pany the profound changes that have occurred in a post-pandemic scenario, highly digitized, 
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with collaborative practices between countries and that will seek greater socio-environmental 
balance. 

In Latin America, there is a growth in the search for evaluative actions to strengthen and 
improve public policies adopted in the region, mainly those related to confronting social 
vulnerabilities. However, the scenario demands the development of professional capacities of 
governments and an increase in the role of civil society in monitoring and control. The evolu- 
tion and internalization of evaluation processes contribute to the development of robust and 
comprehensive methodologies. Since the 1980s, the search has grown for models capable of 
combining quantitative and qualitative information (mixed methods), with a view to improv- 
ing evaluation and making it more realistic with regard to the complexity of the analyzed 
policies. The search for new robust and comprehensive evaluation methodologies is essential 
for the development and evolution of evaluation activities, providing managers with concrete, 
real and applicable data. The search for transparency and effectiveness in the management of 
projects and programs minimizes the misuse of public money and helps in the fight against 
systemic corruption. In a scenario of freezing public resources and changing epidemiological 
profiles of the population, the systematic use of evaluation of public policies and programs 
can contribute to the rational use of human, financial and material resources by improving 
and strengthening the practices carried out at all levels of government. Therefore, the aim is 
to use innovative methodologies while providing professional and scientific training, which 
can go on to support decision-makers in relation to social public policies — especially those in 
health and education. 


References 


Ala-Harja, Marjukka e Sigurdur Helgason. 2014. “Em direção às melhores práticas de avaliação”. Revista do 
Serviço Público v. 51 n. 4, pp. 5—60. https://doi.org/10.21874/rsp.v5 114.334. 

Bilella, Pablo, D. Rodriguez, et al. 2016. “Diretrizes para avaliação para a América Latina e o Caribe”. Bue- 
nos Aires. <www.deval.org/fileadmin/Redaktion/Bilder/2016_FINAL_ESTANDARES_DIGITAL_ 
PORTUGUES. pdf>. 

Bloch, Carter, et al. 2014. “Developing a methodology to assess the impact of research grant fund- 
ing: A mixed methods approach”. Evaluation and Program Planning v. 43, pp. 105—117. https://doi. 
org/10.1016/j.evalprogplan.2013.12.005. 

Ceneviva, Ricardo e Marta Ferreira Santos Farah. 2012. “Avaliação, informação e responsabilização no 
Setor Público”. Revista de Administração Pública v. 46 n. 4. Rio de Janeiro, RJ:993—1016. <https:// 
bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/7121>. 

CIDACS. 2021. “Centro de Integração de Dados e Conhecimentos para Saúde”. Quem somos. Disponível 
em: <https://cidacs.bahia.fiocruz.br/sobre/quem-somos/>. 

Cohen, Ernesto e Rolando Franco. 2008. Avaliação de projetos sociais. 8. ed. Rio de Janeiro: Vozes. 

Costa, Antonio Firmino da, et al. 2013. “Mixed-methods evaluation in complex programmes: The national 
reading plan in Portugal”. Evaluation and Program Planning v. 39, pp. 1-9. https://doi.org/10.1016/j. 
evalprogplan.2013.02.001. 

Costa, Frederico Lustosa da e José Cezar Castanhar. 2003. “Avaliação De Programas Públicos: Desafios 
Conceituais E metodológicos”. Revista De Administração Publica v. 37, n. 5. Rio de Janeiro, RJ, pp. 969- 
992. <https://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/6509>. 

Cotta, Tereza Cristina. 1998. “Metodologias de avaliação de Programas e Projetos Sociais: Análise de 
Resultados e de Impacto”. Revista do Serviço Público v. 49, n. 2, pp. 103—124. https://doi.org/10.21874/ 
rsp.v49i2.368. 

Cotta, Tereza Cristina. 2001. “Avaliação Educacional e políticas Públicas: A Experiência do Sistema Nacio- 
nal de Avaliação da Educação Básica (Saeb)”. Revista do Serviço Público v. 52, n. 4, pp. 89-111. https:// 
doi.org/10.21874/rsp.v52i14.316. 

Cruz, Marly. M. 2015. “Avaliação de políticas e programas de saúde: contribuições para o debate”. In 
Caminhos para análise das políticas de saúde edited by Ruben Araujo de Mattos e Tatiana Wargas de Faria 
Baptista. Porto Alegre: Rede Unida, pp. 285-317. 


386 


Framework for Public Policy Evaluation 


Derlien, Hans-Ulrich. 2001. “Una comparación Internacional De La evaluación De Las políticas Públicas”. 
Revista do Serviço Público v. 52, n. 1, pp. 105-123. https://doi.org/10.21874/rsp.v5211 .302. 

EVALPARTNERS. 2015. Global Evaluation Agenda 2016-2020. <www.evalpartners.org/sites/default/ 
files/documents/EvalAgenda2020.pdf>. 

EVALPARTNERS. 2016. Global evaluation agenda. <www.evalpartners.org/global-evaluation-agenda>. 

Faria, Carlos Aurélio P. 2005. “A politica da avaliação de políticas públicas”. Revista Brasileira de Ciências 
Sociais v. 20, n. 59, pp. 97-109. https://doi.org/10.1590/S0102-69092005000300007. 

Feinstein, Osvaldo. 2019. “Dynamic evaluation for transformational change”. In Evaluation For Transforma- 
tional Change edited by Rob D. van den Berg, Cristina Magro e Silvia Salinas Mulder, pp. 17—31. Exeter, 
United Kingdom: International Development Evaluation Association (IDEAS). 

FGV CLEAR. 2021. Centro de Aprendizagem em Avaliação e Resultados para o Brasil e a África Lusófona 
(FGV EAESP Clear). <http://fgvclear.org/pt/sobre-o-fgv-clear/>. 

Howlett, Michael, M. Ramesh e A. Perl. 2013. Politica pública — seus ciclos e subsistemas: uma abordagem inte- 
gral. Rio de Janeiro: Elsevier. 

Instituto Brasileiro de Geografia e Estatística — IBGE. 2018. Indicadores ODS <https://ods.ibge.gov.br>. 

Mokate, Karen Marrie. 2002. “Convirtiendo el ‘monstruo’ en aliado: la evaluación como herramienta de 
la gerencia social”. Revista do Serviço Público v. 53, n. 1, pp. 89-134. <https://repositorio.enap.gov.br/ 
handle/1/1964>. 

Oliveira, Lilian Ribeiro de. 2018. Políticas públicas sociais: avaliação e integração entre assistência social 
e saúde publica no estado de São Paulo [tese]. Ribeirão Preto: Faculdade de Economia, Administ- 
ração e Contabilidade de Ribeirão Preto da Universidade de São Paulo. Doi:10.11606/T.96.2019. 
tde-08042019-093527. 

Oliveira, Lilian Ribeiro de e Claudia Souza Passador. 2019. “Theoretical essay on public policy evalua- 
tions”. Cadernos EBAPE. BR v. 17, pp. 324-337. https://doi.org/10.1590/1679-395169657. 

Organisation for Economic Co-Operation and Development — OECD. 2002. Glossary of key terms in evalu- 
ation and results-based management. Paris: OECD. 

Organisation for Economic Co-Operation and Development — OECD. 2016. Measuring distance to 
the SDGs targets: A pilot assessment of where OECD countries stand. <www.oecd.org/std/oecd- 
measuring-distance-to-the%20sdgs-target-pilot-study-web.pdf>. 

Organização das Nações Unidas — ONU. 2015. Transformando nosso mundo: a agenda 2030 para o desen- 
volvimento sustentável. <https://nacoesunidas.org/pos2015/agenda2030/>. 

Organização das Nações Unidas — ONU. 2016. Global Evaluation Agenda: Eval2016—2020. [s.l]: 
EvalPartners. 

Patton, Michael Quinn. 1999. Utilization-focused evaluation in Africa. <http://gametlibrary.worldbank. 
org/FILES/284_Utilisation%20Focused%20evaluation%20in%20Africa.pdf>. 

Picciotto, Robert. 2015. “Democratic evaluation for the 21st century”. Evaluation v. 21, n. 2, pp. 150-166. 
https://doi.org/10.1177%2F1356389015577511. 

Picciotto, Robert. 2019. “Is evaluation obsolete in a post-truth world?”. Evaluation and Program Planning v. 
73, pp. 88-96. https://doi.org/10.1016/j.evalprogplan.2018.12.006. 

Picciotto, Robert. 2020. “Towards a complexity framework for transformative evaluation”. Journal of Mul- 
tiDisciplinary Evaluation v. 16, n. 35, pp. 54-76. <https://journals.sfu.ca/jmde/index.php/jmde_1/ 
article/view/643>. 

Picciotto, Robert. 2021. “Evaluation as a social practice: Disenchantment, rationalities and ethics”. Evalua- 
tion and Program Planning v. 87, p. 101927. https://doi.org/10.1016/j.evalprogplan.2021.101927. 

Programa das Nações Unidas para o Desenvolvimento — PNUD. 2016. Objetivos de desenvolvimento 
sustentavel. <www.pnud.org.br/ods.aspx>. 

Queiroz, Lucia de Fatima Nascimento de e Mauro Guilherme Maidana Capelari. 2020. “Condições para 
avaliação de resultados em políticas públicas: um debate sobre o papel da institucionalidade”. Revista 
de Administração Pública v. 54, n. 3, pp. 433-447. <https://bibliotecadigital.fgv.br/ojs/index.php/rap/ 
article/view/81261>. 

Ramos, Marília P. e Leticia M. Schabbach. 2012. “O estado da arte da avaliação de políticas públicas: con- 
ceituagao e exemplos de avaliação no Brasil”. Revista de Administração Publica, Rio de Janeiro v. 46, n. 5, 
pp. 1271-1294. https://doi.org/10.1590/S0034-76122012000500005. 

Rossi, Peter H. e James D. Wright. 1984. “Evaluation research: An assessment”. Annual Review of Sociology 
v. 10, pp. 331-352. https://doi.org/10.1146/annurev.so. 10.080184.001555. 

Sano, Hironobu e Mario Jorge França Montenegro Filho. 2013. “As técnicas de avaliação da Eficiência, efica- 
cia e Efetividade na gestão Publica e sua relevância para o Desenvolvimento Social e das ações públicas”. 
Desenvolvimento Em Questão v. 11, n. 22, pp. 35—61. https://doi.org/10.21527/2237-6453.2013.22.35-61. 


387 


Lilian Ribeiro de Oliveira and Claudia Souza Passador 


Scriven, Michael. 1991. Evaluation thesaurus. 4. ed. Newbury Park, CA: Sage. 

Silveira, S. E R., et al. 2013. “Políticas públicas: monitorar e avaliar para qué?” In Políticas públicas, gestão e 
sociedade edited by M. A. M. Ferreira and L. A. Abrantes. Viçosa: Triunfal, pp. 301-327. 

Sophie Project. 2012. Evaluating the impact of structural policies on health inequalities and their social 
determinants, and fostering change (SOPHIE). <www.sophie-project.eu/project.htm>. 

Sophie Project. 2015. Conclusions. <www.sophie-project.eu/pdf/conclusions.pdf>. 

Spink, Peter. 2001. “Avaliação democrática: propostas e prática”. Fundamentos de Avaliação n. 3, pp. 7—25. 
<www.abiaids.org.br/_img/media/colecao%20fundamentos%20avaliacao%20N3.pdf>. 

Trevisan, Andrei Pittol e Hans Michael Van Bellen. 2008. “Avaliação de políticas públicas: uma revisão 
teórica de um campo em construção”. Revista de Administração Pública v. 42, n. 3, pp. 529-550. https:// 
doi.org/10.1590/S0034-76122008000300005. 

Valle-Cruz, David, et al. 2020. “Assessing the public policy-cycle framework in the age of artificial 
intelligence: From agenda-setting to policy evaluation”. Government Information Quarterly v. 37, n. 4, 
pp. 101509-101520. https://doi.org/10.1016/j.giq.2020.101509. 

Vedung, Evert. 2009. Public policy and program evaluation. New Brunswick, NJ: Transaction. 

Vedung, Evert. 2010. “Four waves of evaluation diffusion”. Evaluation v. 16, n. 3, pp. 263-277. https://doi. 
org/10.1177%2F1356389010372452. 

Weiss, Carol H. 1998. “Have we learned anything new about the use of evaluation?” American Journal of 
Evaluation v. 19, n. 1, pp. 21-33. https://doi.org/10.1016/S1098-2140(99)80178-7. 

Weiss, Carol H. 1999. “The interface between evaluation and public policy”. Evaluation v. 5, n. 4, pp. 468— 
486. https://doi.org/10.1177%2F135638909900500408 


388 


30 


POLICY EVALUATION 
INSTRUMENTS 


Tools for Learning 


Antje Witting 


This chapter encapsulates current thinking on the kinds of tools that can be deployed in order 
to enhance learning and evidence use in a policy-making context. It starts with an overview 
of the key concepts before introducing contemporary theories on policy learning that utilize 
them. Crucial here are the questions What tools are available? and How do you know which to 
use when? Underlying these questions is the assumption of policy-oriented learning. In other 
words, it is being assumed that an average person, not a super human being, actively searches 
for information to better understand a policy issue and possible solutions. However, the theories 
and tools that are being reviewed for this chapter do pay attention to the fact that not all human 
beings share the same training, experiences or cognitive capacity and that their capacity to use 
these resources also depends on contextual factors, such as the nature of the issue. Metaphorical 
examples that outline the tools’ potential uses at the different stages of the policy process are used 
to explore answers to the questions stated in this chapter. 


Introduction 


As we have seen in earlier chapters, policy formulation and implementation rely heavily on 
public officials who evaluate the success or failure of a specific policy design (the content and 
structural logic of public policy) to integrate these insights at a practical and political level. To 
ensure their findings are actually incorporated at a political and practical level, evaluators need to 
understand who wants the results and for what purpose. Evaluators also need to know whether 
the evaluation procedures are practical (given the time, resources and expertise available), if the 
evaluation is being conducted in a fair and ethical way and whether approaches at each step 
are accurate, given stakeholder needs and evaluation purpose. In addition, evaluators also rely 
on their skills to verify the scientific quality of their research and integrate their insights at the 
political and practical level. Studies show that policy-relevant advice can dominate a policy 
choice if it is salient (or relevant), credible and legitimate (Cash & Belloy 2020; Cash et al. 2003). 

The notion “policy capacity” can be used to conceptualise this interplay of analytical, mana- 
gerial and political knowledge and skills of individuals engaged in the policy process, such as the 
public officials evaluating a policy design, and organisational and system level capacities, such as 
effective inter- and intra-organisational communication (Painter & Pierre 2010). Whether an 
increase in policy capacity ultimately also increases the likelihood that these individuals learn 
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about policy failures and success stands to debate. The contribution presented here explores this 
question at an individual level. First, it elaborates on the definition of policy capacity at an indi- 
vidual level. Second, it explores the notion that the learning tools associated with collaborative 
settings drive the development of policy capacity at an individual level. 


Learning About Policy Failures and Successes 


At the policy evaluation stage, public officials search and adapt to information about the suc- 
cess or failure of a policy design (Howlett et al. 2009). This search takes many shapes, as de 
Oliveira and Passador elaborate on in their chapter. Generally speaking, individuals engaged in 
an evaluation usually pursue scientific and technical knowledge about shortcomings in the con- 
tent and structural logic of public policy. Scholars observe that this is hardly ever a linear process. 
Definitions of complex policy issues and potential political solutions are constantly renegotiated. 
Public officials are assumed to be rational in their pursuit of knowledge but are limited in their 
ability to acquire and use new insights (Sabatier 1988). 

Over time, public officials are expected to develop analytical capacity: that is, the knowledge 
and skills needed to conduct and communicate policy analyses and evaluations, such as environ- 
mental scanning, trends analysis and forecasting methods, theoretical research, statistics, applied 
research and modelling and evaluation of the means of meeting targets or goals (Wu et al. 2017). 
Furthermore, public officials need to hone their political capacities to understand their funda- 
mentally political environment (Peters 2015; Goodsell 2011), including insights about positions, 
interests, resources and strategies of key players involved in the formulation and implementa- 
tion of the policy and the practical implications of proposed policy changes (Wu et al. 2010). 
Finally, officials need to ensure that proposals can be implemented in practice through effective 
planning, staffing, budgeting, delegating, directing and coordinating. A lack of such managerial 
capacity may result in policy recommendations and eventually decisions that are sound in theory 
or principle but fall apart in practice (Wu et al. 2015). 

To avoid ineffective or inappropriate policy monitoring and/or feedback processes and 
structures, public officials must develop sufficient analytical, managerial and political capacities 
(Howlett 2009). In addition, they must also develop the interpersonal relationships needed to 
communicate their insights and the intended course of action horizontally, within and across 
policy domains; to understand the requirements for a specific policy and its effects from different 
perspectives; to reduce duplication; and to identify possible unknowns. In particular, non-elected 
officials are expected to ensure that policy and political expertise concerning the formulation, 
implementation and evaluation of a policy design filters from the top level through to the “street” 
level of the organisation and vice versa. This requires vertical communication (Torfing et al. 
2012). In addition, it requires outward communication, as public officials seek support in their 
political battles (Goodsell 2011). Policy analysts tend to define the emerging systems as policy 
subsystems (Sabatier 1988). Policy subsystems usually involve a broad set of individuals, such as 
officials and politicians within government managing the policy in question, a policy-research 
community supporting them and members of the public with an interest in a government’s policy 
record (Peters 2015). Policy subsystems form at earlier stages of the policy cycle and constitute 
the context in which individuals acquire, make sense of and disseminate information at the policy 
evaluation stage (Sabatier 1988). The political nature of judgments about policy success and fail- 
ure also implies that such assessments are rarely unanimous (Howlett 2009). 

Scholars in the field of public policy analysis observe that individuals in a policy subsystem who 
share a similar perception of an issue and preference for a specific solution cluster into advocacy 
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coalitions (Jenkins-Smith et al. 2014). Regular and trust-based interactions in an advocacy coali- 
tion bias judgments as to the nature of the problem and the effectiveness and beneficiaries of 
specific solutions and subsequently help rationalise prior steps in the policy process (Weible et al. 
2010). These biases may manifest as self-interested, corrupt or clientelistic behaviour (see Capano & 
Howlett 2020) or “placebo policies” (McConnell 2020, p. 965). Mukherjee et al. (2021) argue that 
the difference in their policy capacity determines whether policy officials can rationally assess the 
risks of policy failure or perverse outcomes. The available evidence base highlights the importance 
of developing policy capacity at an individual but also organisational and system level. 


Policy Capacity at an Individual Level 


More specifically, policy analysts observe that individuals in a policy subsystem develop deeply 
rooted cognitive filters that help them effectively address policy issues in familiar situations 
by using heuristic processing (Jenkins-Smith et al. 2014). Policy analysts also observe that 
individuals with high analytical capacity are more actively engaged in the search for infor- 
mation (Weible et al. 2011), not accounting for factors that impact the actual access and 
means to analyse information (see, e.g., Wu et al. 2017). Witting (2017) proposes that four 
categories of learners can be distinguished along these two cognitive dimensions (see Figure 
30.1): novices who utilise basic logical reasoning skills but generally lack specialised knowl- 
edge to solve task-related problems effectively by using heuristic processing; specialists with 
basic logical reasoning skills, who engage in routinised tasks and have specialised knowledge 
applicable only within a particular environment; advisors with advanced logical reasoning 
skills who accumulate specialised knowledge applicable only within a particular environment; 
and scientists with extremely potent abstract reasoning skills and working memory (possess- 
ing academic abilities that are considered general knowledge in most cultures, such as historic 
knowledge, literacy and numeracy). This predisposition increases the potential that they can 
use reasoning to identify optimal solutions. 


Advanced 


Inclined towards (Type 2) analytical processing 
Inclined towards (Type 1) heuristic processing 


Low Level of Specialized Knowledge High 


Logical Reasoning 


Basic 


Figure 30.1 Learner Types 


Source: Witting 2017 (reprinted with permission) 
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Looking Beyond a Person’s Analytical Capacity 


The issues for which subsystem members with different analytical capacities seek optimal solu- 
tions are often too complex to be fully comprehended by one individual alone. They reach out 
and connect with one another. When confronted with an ambiguous evidence base, individuals 
tend to connect to others who advocate a similar approach or perspective. For example, a study 
comparing the flow of information during a post-flood evaluation finds that the engaged indi- 
viduals do not rely excessively on one person to distribute information between departments or 
private sector partners (Witting et al. 2019, 2020). Depending on their analytical and manage- 
rial skills, individuals take different positions in a policy subsystem. This affects their political 
capacities. Theoretically, several positions can be distinguished using measures of betweenness 
(Howlett et al. 2017). For simplicity, this discussion focuses on three commonly cited positions. 

Entrepreneurial position at a coalition’s core: Individuals with specialised knowledge may 
find themselves in an entrepreneurial position at the core of an advocacy coalition when they are 
invested in the process long term, and their expertise is recognised by their peers (Weible et al. 
2004). Policy entrepreneurs are expected to favour knowledge that supports their argument as 
they seek to attract sufficient support to ensure their preferred outcome. From this, it can be 
deduced that the learning space of an entrepreneur constitutes the most densely connected and 
homogenous clusters within a policy subsystem, which can also be defined as the core of an 
advocacy coalition (Weible 2008). Knowledge circulates within this group. Ingold (2011) finds 
that this group includes representatives of government agencies or powerful interest groups. 
Crow (2010) observes entrepreneurs to be the most effective when they have accumulated more 
technical or managerial expertise over time. Not all entrepreneurs are interested in changing the 
status quo (Mintrom & Norman 2009). Christopoulos (2006) distinguishes incremental from 
opportunistic strategies. Incrementalists use their capacity and social capital to build a consensus 
around the status quo and further institutionalise their advisory position. Opportunists have to 
first draw attention to their expertise, and they exploit windows of opportunity to do so. Both 
strategies are at play in policy-making contexts. 

Brokerage positions at a subsystem’s core: In contrast to entrepreneurs, policy brokers’ prin- 
ciple concern is to provide cues for analytical reasoning and to mediate conflict between com- 
peting coalitions (Sabatier 1988, p. 133). They are assumed to trigger “new ideas concerning, 
e.g., causal relationships and policy instruments” (Sabatier 1988, p. 159). Scientists, journalists 
and civil servants tend to occupy this position at the periphery of each coalition (Ingold & 
Varone 2012). They engage in dialogue and deliberation with individuals in opposing coalitions 
(Ingold 2011). They are central members in the network but are only intermittently involved, 
or involved for a short period, and they do not regularly engage in coalition-related activities 
(Weible 2008). Thus, brokers play a less integrative role within a coalition but drive collabora- 
tions on a subsystem level. This evidence supports the argument that scientists and journalists 
who interfere in a specific discussion can, for a short period, create bridges via which individuals 
in opposing coalitions can then send and receive cues for analytical reasoning (Sabatier 1988). 

Periphery of a coalition: To borrow a definition found in Mintrom and Norman (2009), 
“an advocate is an individual who is comfortable working within established institutional 
arrangements” (ibid. 650). They hold authority or potential for authority to enforce and 
monitor policy design and implementation, but they are not interested in upsetting the status 
quo. Advocates may not be central to a coalition’s core (like entrepreneurs) or the subsystem 
as a whole (like brokers), but they are nevertheless skillful leaders with the legal author- 
ity, financial means, social capital or knowledge to impact individual and social learning. 
Montpetit (2011) argues that this can include scientists who share their expertise with policy 
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entrepreneurs. They are targeted by policy entrepreneurs who seek sufficient support to gain 
subsystem supremacy. Members of the group who receive cues for analytical reasoning from 
their exogenous environment may approach brokerage positions as they seek to make sense 
of their observations (Weible 2008). It is thus expected that advocates constitute the loosely 
connected periphery of a coalition. 


Drivers and Barriers of Evaluative Judgments at an Individual Level 


Based on the type of learner and their position in the policy subsystem, we can draw conclusions 
about their capacity to detect biases in their evaluative judgments (see Table 30.1). 

Scientists take on brokerage roles within a specific network, but rarely engage in one par- 
ticular process for a long period of time. Consequently, they do not have the political skills or 
enough history in common with other network members for strong relationships to manifest 
and to ensure that their perspective is heard. One communication barrier is that their analytical 
reasoning process is slow in comparison to that of advisors, whose specialist knowledge of the 
issue domain enables them to engage in heuristic reasoning, much like specialists. 

Advisors and scientists have a similar academic background, but the advisor acquires more 
specialist knowledge through experience than the scientist. In contrast to specialists, advisors are 
trained in critical thinking. Like specialists, they rely on their innate belief structure to quickly 
arrive at satisfying outcomes. This can result in analytical biases (heuristic processing) as they 
are involved in the policy subsystem for longer periods of time. This increases the likelihood 
of judgmental biases forming and makes it less likely for them to inhabit a brokerage position. 
Unlike specialists, advisors find it easier to detect cognitive conflicts, using their advanced logical 
reasoning skills, and to adapt their routinised decision behaviour. 

Compared to novices, specialists acquire — through experience — sufficient knowledge 
about specific issues and tools to satisfactorily address such issues. Like novices, they lack 
advanced training in logical reasoning and are likely to rationalise their judgmental errors. 
Removing specialists from their domain of expertise or radically changing attributes of the 
domain would render them equal to a novice. As long as the characteristics of the new domain 
are similar, cues that link the situations can be used to optimise their response. It is thus not 
surprising to find specialists at a coalition’s core, given their specialist knowledge and social 
capital for a particular policy context. However, it is expected that specialists are no more than 
novice advocates when confronted with novel problems and tasks. Specialists use information 
to rationalise their belief bias. Hence, a direct link between scientists in brokerage positions 
and specialists in entrepreneurial positions is not beneficial. The latter has to be able to think 
on their feet and lacks the time to review scientific reports, while the former is less invested 
in a particular cause and more invested in producing verifiable and generalisable material that 
describes the bigger picture. 

Novices are the slowest learners of them all, due to a lack of specific knowledge and novel 
problem-solving ability — assuming they just recently joined the policy subsystem. Since they 
lack the expertise to successfully engage in heuristic reasoning, they are more inclined to process 
information analytically. Given their basic academic training and lack of specialist knowledge, 
they are the most likely to inhabit the position of an advocate. Depending on their position 
in a policy subsystem, novices will be more likely to acquire domain-specific or generalisable 
knowledge. In other words, depending on their education and contact with scientist brokers or 
specialist entrepreneurs, novice advocates develop a narrow or broad understanding of the policy 
problem. However, keep in mind that they may have policy capacities that suit another policy 
context and thus can take a brokerage position between policy subsystems. 
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Table 30.1 Capacities of the Different Learner Types 


Analytical Capacities Managerial Capacities Political Capacities 
Novice Basic logical reasoning Highest when person occupies Capacities to attract 
skills, lacks subsystem- an entrepreneurial position advocates from another 
specific knowledge in another subsystem policy subsystem to an 
advocacy coalition 
Specialist Specialised knowledge, Highest when person occupies Capacities to attract 
lacks advanced training an entrepreneurial position advocates to an 
in logical reasoning in an advocacy coalition advocacy coalition 
Advisors Advanced logical reasoning Highest when person occupies Capacities to attract 
skills and specialised an entrepreneurial position advocates to an 
knowledge in an advocacy coalition advocacy coalition 
Scientist Advanced logical reasoning Highest when person occupies Capacities to mediate 
skills, lacks subsystem- a brokerage position in a conflicts between 
specific knowledge policy subsystem advocacy coalitions 


In sum, managerial and political skills can enforce judgmental errors when the individuals 
lack the analytical skills necessary to detect biases in their thinking. More specifically, the 
discussion implies that individuals in entrepreneurial positions would be using their capac- 
ity and social capital to build a consensus around the status quo and further institutionalise 
their advisory position. Resulting, for example, in an evaluative judgment that overestimates 
a point of view that is overly present in their everyday communications. More importantly, 
individuals in an entrepreneurial position would have the managerial and political skills to 
activate sufficient support for what they judge to be the right way forward, even if it is not 
reflecting a rational assessment of the evaluated phenomenon. The more important are tools 
with which to support the development of a person’s analytical capacity at an organisational 
and system level, in particular if said person has sufficient analytical and political capacities to 
bias policy evaluations. 


Tools to Develop an Individual’s Capacity 


If organisations or systems are to build an individual’s, and subsequently their capacity for, 
learning through building individual policy capacity, they rely on all the individuals to be 
able to send and receive feedback that challenges their evaluative judgments (Ansell & Gash 
2008). In other words, to avoid evaluative judgments, they need institutional settings that 
guide individuals into brokerage positions where they are exposed to different ideas and can 
weigh the pros and the cons as well as consider the practicalities and political relevance of 
these ideas. 

Scholars specifically observe in their empirical studies that policy-relevant knowledge typically 
grows from interdisciplinary collaborations, as collaborations can reduce the transaction costs of 
communication between individuals with divergent mind-sets (Beck 2012, Cash et al. 2003; 
Cash & Belloy 2020) or at different centres of decision-making (Andersson & Ostrom 2008). 
Wood and Gray (1991) define a collaboration as a group of autonomous members of a policy 
subsystem, who, in an interactive process, act or decide on issues related to that domain, following 
agreed-upon procedures (Wood & Gray 1991, p. 146). Ansell and Gash (2008) adopt a narrower 
definition. They define a collaboration as a governing arrangement in which one or more public 
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agencies directly engage non-state stakeholders in a collective decision-making process that is 
formal, consensus oriented, and deliberative and that aims to make or implement public policy or 
manage public programs or assets. Their definition describes arrangements which are initiated by 
public agencies or institutions and include non-state actors. Participants engage directly in deci- 
sion-making and are not merely “consulted” by public agencies. The forum is formally organised 
and meets collectively. The forum aims to make decisions by consensus (even if consensus is not 
achieved in practice), and the focus of collaboration is on public policy or public management. 
Collaborative structures, such as the one described by Ansell and Gash (2008), are often nested 
within vertical bureaucracies, if not coordinated by them (Emerson & Nabatchi 2015a). While 
the types of participants in collaborations may vary from definition to definition, collaborative 
settings exhibit common design features, introduced next under the headings Diversity in Rep- 
resentation, Balanced Power, Interactive Dialogue and Transparent Procedures. 


Diversity in Representation 


Diversity of the represented expertise and interests ensures that all stakeholders who can impact 
an outcome are aware of the issue (Cash et al. 2003). Diversity is seen as boosting learning by 
providing for different points of view and knowledge types (Heikkila & Gerlak 2015). This is 
by far a new notion. Public officials in Germany started as early as in the nineteenth century 
to enhance their policy capacity by inviting scientific expertise to determine the objectives and 
indicators of a successful policy design. Out of this rationale, the first departmental research 
institutes (RFEs) emerged in Germany (WR 2004). For example, the growing need of (central) 
governmental for disease control led to the establishment of the Königlich preußisches Institut 
für Infektionskrankheiten in 1891, which is today known as Robert Koch Institute (RKI) — a 
key player in combating the current SARS-CoV-2 pandemic. 

RFEs are nowadays publicly funded institutes that either design and initiate policy evalu- 
ations themselves or reward universities, research institutions, expert organisations or private 
enterprises with contracts or grants to do so, following a public tender. More specifically, RFE 
advisors occupy an entrepreneurial position. They systematically enable the integration of sci- 
entific policy advice in the policy process in three steps. In the first step, this contribution is 
made through integrated work at the research level. RFEs implement interdisciplinary research 
that is geared to practical problems in a policy field, either through RFE advisors or through 
commissioning scientists. In the second step, departmental research institutions, through their 
advisors, integrate the diverse requirements of practice with the options for action of the various 
scientific disciplines. In this context, they promote a practice-oriented formulation of research 
questions through integrated work on the one hand, and on the other hand, they filter scientific 
research results according to their relevance for the practice. In a final step, departmental research 
institutions exploit scientific findings via formats tailored to target groups, such as advisory in 
the department. Each of these steps tends to be led by a specialist unit in which RFE advisors 
cooperate on a day-to-day basis (Bocher & Krott 2010). The divisional organisation of the work, 
according to the sectoral principle, favours subject specialisation (e.g., better understanding of 
practical problems and subject expertise in the policy field). 

Subject specialisation lowers the transaction costs of communication between representa- 
tives of science and policy praxis in a policy subsystem, thus increasing the likelihood that new 
insights are translated into policy (B6cher 2012). However, in 1997 and 2005, the Science 
Council (WR) evaluated the scientific quality of RKI research and criticised a lack of diversity 
(WR 1997, 2005). The WR encouraged the RKI to focus and develop its specialised expertise 
and keep abreast of the latest scientific developments in order to fulfil its official tasks properly. 
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In other words, it nudged the institute toward scientists at a brokerage position in a policy sub- 
system — a nudge commonly found in WR evaluation reports of departmental research institutes 
(WR 2007, 2010, 2013). As a result of the WR evaluations, the Scientific Advisory Board at 
the RKI was established in agreement with the Federal Ministry of Health. The members of the 
Scientific Advisory Board are appointed by the president of the RKI in consultation with the 
Federal Ministry of Health for four years at a time and are to accompany the RKI in carrying 
out its professional work at an internationally competitive level. The Scientific Advisory Board 
regularly meets at least twice a year and also in ad hoc meetings as required. The Advisory Board 
regularly comments on the professional and scientific performance of the RKI, participates in 
an advisory capacity in the development of medium- and long-term research goals, promotes 
trans- and interdisciplinary cooperation and provides expert advice on request. Furthermore, 
it was agreed that the Scientific Advisory Board would appoint international expert panels to 
regularly evaluate the scientific work at the RKI on specific topics and to make recommenda- 
tions for development. All in all, these changes are a blueprint for RFEs that also received the 
WR nudge as a result of their evaluation. 


Balanced Power 


Whether these regular meetings alone are sufficient to combat tendencies to underestimate or 
even ignore the input or expectations of individuals or groups and arrive at faulty evaluative 
judgments has yet to be established. At first glance, interim feedback that the WR published in 
2021 on the institute’s performance during the SARS-CoV-2 pandemic suggests this to be the 
case. While we await the next evaluation report from the WR, first lessons can be extracted 
from the literature on participatory and collaborative procedures. In their review of this litera- 
ture, Ansell and Gash (2008) observe that procedural fairness correlates highly with the levels 
of interpersonal trust among individuals and with the amount of learning and social capital 
generated within a collaboration. Facilitative leadership may be needed to reach consensus 
on the procedures of a policy evaluation — in particular when multiple parties with conflict- 
ing epistemological and ontological views come together to decide on the research questions, 
study methods and scientists and analysts who will implement the research. The aim is to design 
procedures that increase transparency, which can build public trust in authorities and their 
administrative procedures — a point that Kovač and Nihit approach from different perspectives 
in their chapters. 

The point made in this chapter is that successful leadership can ensure that all the involved par- 
ties consider the agreed-on processes and responsibilities to be fair. Leadership can also use mod- 
eration techniques to balances the power distribution in an arrangement. For example, ensure that 
all parties are heard. An equal power distribution increases the incentive to participate — a critical 
variable in the success or failure of a collaboration. Other critical variables are the development of 
a shared understanding of what they can collectively achieve and small wins that assure everyone 
involved that they are heading in the agreed-upon direction (Ansell & Gash 2008). 

Presence and commitment are also relevant. Collaborations in which all stakeholders who 
could be affected by or have a vested interest in the issue are fairly represented increase the legiti- 
macy of their decisions (Emerson & Nabatchi 2015b). For example, Jasanoff (2010) observes that 
recommendations that emerge from collaborations between a narrow elite of experts who share 
the commitment to follow scientific rules and norms fail to convince a wider public. Through 
interactive dialogue and communication, trust can be built, mutual learning and orientation 
towards the common good can emerge and mutual gains can be identified; all of these factors 
are assumed to lead to effective decision-making (Cash & Belloy 2020). 
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Interactive Dialogue 


These dynamics are supposed to intensify over time, requiring an interaction of stakeholders that 
goes well beyond a one-time experience (Emerson & Nabatchi 2015b). Direct dialogue is nec- 
essary for stakeholders to identify opportunities for mutual gain and build trust (Ansell & Gash 
2008). Renn (2006) argues that collaborations must design mechanisms that go beyond inten- 
tion and actually enable the incorporation of expertise, as well as different interests and values. 
In other words, a one-day workshop is not sufficient a collaboration to build policy capacity. Pita 
et al. (2010) also highlights variations in the perceived levels of participation, ranging from pas- 
sive (informed) participation to functional participation (stakeholders are actually involved). In 
theory, successful collaborations also actively employ measures to empower disadvantaged stake- 
holders and subsequently ensure that all the interests are sufficiently represented in the interac- 
tive process (Ansell & Gash 2008). To accomplish this, power imbalances need to be avoided 
during the deliberation when information is shared amongst the diverse stakeholders (Heikkila 
& Gerlak 2015). If (some) stakeholders do not have the capacity, organisation, status or resources 
to participate or to participate on an equal footing with other stakeholders, collaborative efforts 
will be prone to manipulation by stronger actors (Ansell & Gash 2008). For example, advisors in 
an entrepreneurial position can dominate novice advocates who may be in a position to transfer 
impulses from another policy subsystem. This power imbalance subsequently increases the risk 
that the judgmental biases of a few lay the foundation of policy failure that affects all. 


Limits of Developing an Individual’s Policy Capacity 
in a Collaborative Setting 


Scholars also caution that the management of collaborations is resource intensive, which doesn’t 
always result in positive outcomes (Matczak et al. 2018). The limits of collaborative policy designs 
for developing policy capacity at an individual level can be illustrated with several examples. 

The first example is drawn from an observation of a temporary post-crisis collaboration. Wit- 
ting et al. (2021) deduce from studies of collaborative, participatory governance arrangements 
strategies to systematically identify and evaluate the properties of an individual’s learning space 
that are indicative of a collaborative setting. In a related study of individuals in a collaboration 
that evaluated a devastating flash flood in South-West Germany, Witting et al. (2020) observe, 
by overlaying individual networks, that scientists occupied a fringe position in the otherwise 
well-integrated network of specialists. Furthermore, the evidence presented in this study high- 
lights the time associated with deliberative bridging activities between specialists, such as regular 
inter- und transdisciplinary workshops to monitor the progress of different design aspects. The 
surveyed specialists explained this pattern with a lack of time; in particular after the flash-flood 
event, they faced immense public pressure to deliver an risk assessment and unintentionally 
opted against the integration of new scientific expertise in their communication flow. However, 
they did consider the evidence when the pressure decreased. The evidence also shows that spe- 
cialists were willing but overwhelmed by the task of coordinating their plans and actions across 
administrative levels over a prolonged period of time. Finally, the study finds that this collabora- 
tion profited from an already-existing culture of collaboration in the policy field in which these 
individuals operated (Witting et al. 2019). 

The role of departmental research institutes (RFEs) in assessing the risks of the COVID-19 
pandemic for the German government is an example to illustrate the advantages and shortcom- 
ings of an institutionalised collaborative design. The experiences from the current SARS-CoV-2 
pandemic show the enormous importance of RFEs. In particular, the RKI and the RFE for 
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vaccination and biometrics (Paul Ehrlich Institute) comprehensively engaged at the policy evalua- 
tion stage since the beginning of the crisis and are significantly involved in all relevant crisis teams. 
The prompt transfer of knowledge from scientists to advisors in both RFEs to specialists in their 
respective departments, facilitated through additional resources for collaborative crisis manage- 
ment, are essential for the German government’s ability to quickly assess the usefulness of specific 
policy designs. In its position paper “Impulses from the COVID 19 crisis for the further devel- 
opment of the science system in Germany” from January 2021, the WR acknowledges this fact. 
However, the WR also observes a lack of subsystem-spanning structures that would help all RFEs 
coordinate their contributions with another more intensively to ensure that all crisis-related issues 
are equally recognised (WR 2021). 

Two lessons can be taken away from these two examples for the next crisis ahead. First, spe- 
cialists cannot humanly comply with the task of deliberating evaluative procedures and actions 
across policy subsystems over a prolonged period of time without supportive tools. Second, the 
challenge ahead is to create structures that facilitate cross-subsystem interaction in a crisis while 
also supporting high levels of specialised knowledge transfer and quality assurance within each 
subsystem. 


Conclusion: Evaluating Changes in Policy Capacity 


As stated at the outset of this chapter, the discussion is motivated by the question of whether an 
increase in policy capacity ultimately also increases the likelihood that individuals learn about 
policy failures and success at the evaluation stage. The first part of the chapter elaborated on the 
literature conceptualising the interplay of analytical, managerial and political knowledge and 
skills of individuals engaged in the policy process. It highlights different learner types. The sec- 
ond part reflects on the notion that the learning tools associated with collaborative settings drive 
the development of policy capacity at an individual level. It refers to two examples showing that 
collaborations can establish and mediate communication between specialists at the political level 
and the community of scientists but still lack the capacity needed to rationally scrutinise and 
further develop their insights through external quality assurance. 

The more important are attempts to evaluate the effect of these tools on the development of 
an individual's policy capacity. Hence, this discussion concludes with some thoughts on recent 
methodological developments in the field that allow us to do just that. The majority of the 
empirical examples cited here rely on a metaphorical approach to describing policy capacity 
at the different levels. Metaphorical approaches tend to capture data (for example, through 
stakeholder interviews or a document analysis) concerning what individuals are involved in 
decision-making and the interdependence of a given group without seeking to describe these 
connections in more detail (see, e.g., Cash & Belloy 2020; Beck 2012). As we have seen, these 
studies do not necessarily identify the type of relationship under consideration (e.g., informa- 
tion exchange, formal reporting structure). Researchers often fail to capture details about who 
exactly interacts with whom (via whom), the frequency of these interactions and the specific 
direction or purpose (content) of a given relationship. In contrast, comparatively few of the 
cited studies employ social network analysis to describe governance arrangements (see, e.g., 
Howlett 2017). It seems that network surveys are particularly useful for studying changes in the 
properties of policy subsystems, such as changes of individuals in the bridging positions that 
mediate between otherwise disconnected pockets of communication, and their influence on 
policy development. This level of detail is needed to understand why some advice dominates 
the other in the policy process. However, evaluations of policy subsystems as a whole are time 
consuming and cannot always be justified on the grounds of the utility and feasibility of these 
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types of appraisals. Hence, some scholars focus on parts of these structures, such as an individual’s 
immediate social environment (Witting et al. 2019, 2020). 

The challenge ahead is to intelligently combine the available analytical approaches to study 
the knowledge and skills of individuals in the policy process and the factors that drive or hinder 
their efforts to integrate new insights at a practical or political level. This way, we can gradually 
expand the available evidence base of the effects of the different learning tools on an individual’s 
policy capacity. Ideally, these evaluative practices will be used to evaluate the effect of these tools 
on the development of an individual’s policy capacity. 
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One way of classifying evaluations is based on the location of the evaluation, i.e., external evalu- 
ation, internal evaluation, or hybrid evaluation. This chapter examines whether and how the 
difference in the location of evaluators influences the choice of approaches, tools, and proce- 
dural issues in policy evaluation. Subsequently, it illustrates the differences in these dimensions 
of evaluation through a discussion of external and internal evaluation in the case of the Asian 
Development Bank and multilateral development aid. 


Introduction 


The research on procedural policy tools has shown that a variety of tools help governments 
organize and steer the policy process. These types of tools can be used at all stages of the policy 
process, including agenda-setting, decision-making, implementation, and evaluation. Other 
chapters in this book highlight the use of some of these tools during policy evaluation. These 
include behavioral tools and those that encourage public participation in evaluation, for exam- 
ple. This chapter continues this discussion. 

It must be noted at the outset, of course, that evaluations can be of many different types. For 
instance, they could be administrative, judicial, or political. Further, they can be done by the 
public — based on the policy record of the incumbent or the policy proposal of another politi- 
cal party — in the form of elections. Another useful way of classifying an evaluation is based less 
on who carries it out and more on what is evaluated. In this manner, they can be grouped into 
categories depending on whether they evaluate inputs, processes, or outcomes, for example. 
Evaluations also differ in what time period they cover. Ilustratively, life cycle assessments, which 
are used mainly in the case of environmental decision-making or impact, consider the complete 
biophysical cycle in the evaluation. The different stages might include extraction of resources, 
processing, manufacturing, distribution, use, recycling (if any), and disposal (Lee, Benavides, & 
Wang, 2020). Life cycle assessment has been used, for example, to evaluate the impact of waste- 
water treatment; however, what is excluded or included in the analysis is not standardized and 
may vary significantly based on the evaluator (Yoshida, Christensen, & Scheutz, 2013). 

Another way of thinking about evaluations that is common in the literature is based on the 
location of the evaluators: internal or external. Fundamentally, an evaluation can be conducted 
internally, externally, or through a hybrid design (Hildén, Jordan, & Rayner, 2014; Schoenefeld 
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& Jordan, 2017; Weiss, 1993). An inside, government-driven, or formal evaluation is conducted 
by evaluators ‘internal’ to the government; in contrast, an outside, society-driven, or informal 
evaluation is conducted by evaluators ‘external’ to the government (Mathison, 1991; Vedung, 
2006). Meanwhile, a hybrid design enables both internal and external evaluators to directly or 
indirectly work together in evaluating the policy or program. 

This chapter examines how this difference in the location of the evaluator influences the 
choice of evaluative policy tools. We explore this distinction in some detail to help explain the 
approaches and tools involved in policy or program evaluation and how they vary in subject 
matter and effectiveness, among other aspects. 


Internal and External Evaluation 


As noted earlier, key questions concerning program or policy evaluation have always been ‘who 
should exercise the evaluation and how should it be organized?’ (Vedung, 2006). While this can 
refer to which branch of government or civil society carries out the evaluation, it has proven 
useful in the field to distinguish between these two groups in terms of whether or not an evalu- 
ation is carried out ‘internally’ — that is, organized and conducted by the same body or agency 
or government that creates, decides, or implements policy — or ‘externally’ by some more ‘arms- 
length’ individual or body. 

This is a useful distinction to make when considering policy evaluation as a ‘policy tool’: that is, a 
creation of government used to accomplish some end or goal. In this sense, the choice of the location 
of the evaluator can be seen to be closely linked to the objectives or purpose of the evaluation. 

Generally speaking, an external evaluation is perceived as being more objective than an inter- 
nal one (Barrington, 2005) due to its independence from the agency or institution that creates 
both the policy and its evaluative rubric and processes. It is believed that external evaluators are 
more likely to be forthright in their judgments as to the merits and demerits of a policy as they 
have no or little direct relationship with the subject of the evaluation (Barrington, 2005). On 
the other hand, internal evaluators are thought to be more likely to be constrained by financial 
incentives or professional ties within the creating and evaluating organization (Christie, 2008; 
Sonnighsen, 2000). Thus, they are perceived to be more likely to be subjective in conducting 
the evaluation and/or reporting its findings (Love, 2005). Thus, external evaluation might be 
favored when the primary purpose is accountability. 

As with all policy tools, external evaluations, too, have strengths and weaknesses. External 
evaluators may have a broader perspective and independence, but internal evaluators may have 
higher-information processing capacity and access to specialized information and records, which 
external evaluators may not. External evaluators may also lack tacit knowledge about the program or 
policy in question and be overly reliant on formal structures and processes in setting the parameters 
of their investigations and carrying them out (Barrington, 2005; Levin-Rozalis, 2010). An in-depth 
understanding of the relevant context, processes, and programs also enables internal evaluators to 
tailor the evaluation to meet the needs of the organization (Love, 2005). Thus, internal evaluation 
might be favored when the primary purpose is learning and improvement (Mutch, 2012). 

As a result of these differences, the specific kinds of tools used in different types of evaluations 
also tend to vary. 


Approaches in Evaluation 


That is, a variety of choices and approaches exist for evaluation in public policy. At the outset, as 
mentioned earlier, one can distinguish between input-based, process-based, and outcome-based 
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evaluation, all of which involve different kinds of information and assessment tools when they 
are carried out (Pal, 2005). While an input-based evaluation focuses on the effective use of 
resources to achieve policy objectives, an outcome-based evaluation aims to understand the out- 
comes or impacts created by a program or policy, analyze the extent to which they meet stated 
goals, contrast them with the likely outcomes in the absence of a program or policy, or assess 
the extent to which they are beneficial for different stakeholders (Schalock, 2001). In contrast to 
these first two types, process-based evaluation delves into the manner in which the program or 
policy was formulated, adopted, or implemented and the role of stakeholders therein. 

Arguably, the most common approaches or techniques used in external policy evaluation 
are cost-benefit analysis, multi-criteria decision analysis, life cycle assessment, and causal impact 
evaluation. These may also be used in internal evaluations, but they often supplement formal 
analytical techniques with more participatory or collaborative ones involving stakeholder groups 
or the public. While these are not mutually exclusive, the objectives of the evaluation and the 
effort required for operationalizing these approaches could require more attention to one than 
to another. Cost-benefit analysis, for example, involves an appraisal and comparison of the costs 
and the benefits in monetary terms in order to understand the societal value of a program or 
policy and inform resource allocation (Boardman, Greenberg, Vining, & Weimer, 2017). How- 
ever, such detailed cost information may not always be readily available or accessible. 

Increasingly, scholars have also extended this basic technique to incorporate broader societal 
objectives such as equity and environmental sustainability within the framework (Thomas & 
Chindarkar, 2019). For instance, in the case of ecosystem services, Wegner and Pascual (2011) 
recommend situating cost-benefit analysis within a broader approach to utility to incorporate 
human well-being more holistically in the assessment. The results of such expanded analyses, 
however, are sensitive to the weighting of non-monetary costs or benefits in the assessment and 
result in prioritization of some public values over others, a process which can undermine the 
otherwise ‘technical’ application of this tool (Eliasson & Lundberg, 2012). 

Multi-criteria decision analysis has also gained popularity as an approach for assessing out- 
comes (or even input- and process-based criteria) that are difficult to quantify within a utilitar- 
ian paradigm. This technique can be used as a decision aid (see chapters in that section of the 
book) or as an evaluative one. Illustratively, Lucas, Brooks, Darnton, and Jones (2008) use the 
technique to evaluate policies encouraging pro-environmental behavior in the United Kingdom 
based on multiple criteria such as cost, unintended consequences, impact on competitiveness, 
additionality/deadweight, distribution, and uncertainty. 

Several techniques have been developed to help conduct such multi-criteria evaluations. 
Analytic hierarchy process, for instance, provides a framework for capturing various criteria, 
weighting their relative importance, and assessing a policy (or a policy alternative when applied 
to formulation or decision-making) based on those results. In an application of this particular 
technique, Feng and Zhang (2014) evaluate health service system integration policy in China 
with the aim of facilitating policy learning and improvement. Similarly, on the simpler end 
of the analytical spectrum, a ‘scorecard’ is an oft-used technique to evaluate multiple criteria 
simultaneously. In an example of its use, the Cambridge Conservation Forum develop a popular 
scorecard to assess the success of conservation initiatives, and many similar applications exist 
across a wide range of issue areas (Kapos et al., 2008). While analytic hierarchy process, like cost- 
benefit analysis, requires some skill and training in application, scorecards allow non-experts to 
render their judgments about policy success and failure. 

As many of these examples show, while typically considered as an ex-post activity, program 
or policy evaluation has been increasingly ‘conducted’ ex-ante to indicate the likely performance 
of a policy and assist in decision-making. The advancements in modeling and simulation noted 
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in other sections of this book have contributed significantly to this development. Goumas, 
Lygerou, and Papayannakis (1999), for example, use Monte Carlo simulation to estimate the 
probability distribution of net present value of geothermal energy projects under uncertainty. 
Similarly, Fernandes, Cunha, and Ferreira (2011) employ real options analysis to examine the 
feasibility of investments in energy under risk and uncertainty. 

More broadly, Mundaca, Neij, Worrell, and McNeil (2010) review bottom-up economy- 
energy models that help anticipate the effect of policy alternatives on energy decisions made 
by various actors in the system. In an application from a different policy area, Brown, Tullos, 
Tilt, Magee, and Wolf (2009) propose an integrated dam assessment modeling (IDAM) tool to 
simultaneously analyze and visualize numerous biophysical, geopolitical, and socio-economic 
consequences of dam construction. 

Although these approaches are generic and can be used internally, externally, or in a hybrid 
setup, the analytical expertise and experience necessary for them might not be available. Thus, 
complex designs with high information processing requirements are more likely to be favored in 
internal evaluations than in external or hybrid evaluations. 


Tools for Causal Attribution 


Impact evaluation is primarily concerned with causal attribution of changes in outcomes to the 
program or policy. This approach has witnessed rapid growth in recent decades, and its use has 
also diversified across policy areas and geographies (Sabet & Brown, 2018). Econometrics or 
statistics is a common approach for impact assessment, especially among external evaluators in 
academe and specialized research institutes. Developments in these fields that permit relaxation 
of assumptions concerning the functional form and the distribution of variables have further 
increased its utility for policy evaluation (Imbens & Wooldridge, 2009). 

The key development in impact assessment, though, has been a shift in focus from ana- 
lytical techniques toward research design. Experiments or randomized control trials — which 
strengthen attribution of outcomes to specific interventions — have emerged as the gold standard 
for policy evaluation. As an example, Ferraro and Miranda (2013) randomize the implementa- 
tion of a water conservation information campaign in order to evaluate its effect on household 
behavior and inform program design. Such experiments usually entail a hybrid setup as those on 
the inside have a better understanding of the requirements and the constraints while those on the 
outside possess analytical expertise in research design, randomization, and econometrics. The 
Abdul Latif Jameel Poverty Action Lab (J-PAL), for instance, comprises a network of over 250 
research scholars working closely with governments to conduct randomized impact evaluation 
of interventions to alleviate poverty (J-PAL, 2021). 

If the rollout of the intervention is not randomized, natural or quasi-experiments may 
permit evaluation ‘as if? it were random. A regression discontinuity design, for example, 
is a quasi-experimental setup that facilitates determination of causal effect based on a 
comparison of observations close to a ‘cutoff’ or ‘threshold’ that determines eligibility for 
policy intervention. [llustratively, Cerqua and Pellegrini (2014) use a regression discontinuity 
design to evaluate the effect of a regional policy in Italy that subsidized private firms based 
on a multiple ranking system on employment, investment, productivity, and turnover. An 
instrument variable regression is another technique that is used for causal estimation when 
an experiment is not feasible or not successfully administered. Angrist, Imbens, and Rubin 
(1996), for example, propose the use of this technique for impact assessment under specific 
assumptions and demonstrate its application to examine the effect of veteran status on 
mortality in the Vietnam era. 
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While internal evaluators might be more interested in specific cases, external evaluators 
often use such techniques to generalize across cases. [lustratively, Urge-Vorsatz, Koeppel, and 
Mirasgedis (2007) synthesize evidence on the effectiveness of policy instruments in reducing 
buildings’ greenhouse gas emissions from 60 ex-post evaluation reports across 30 countries. 
More replicable techniques such as systematic reviews and meta-analyses are increasingly used 
for this purpose, especially in health policy. 

Further, a program or policy might have ‘heterogeneous’ effects that vary across policy tar- 
gets. In an illustration of an evaluation addressing this issue, Higgins and Kanaroglou (2016) 
use latent class clustering to account for the diversity of station areas in the case of transit- 
oriented development in the Toronto region. While the specific techniques might differ, tools 
to assess such heterogeneous effects could be employed in internal evaluations too: for example, 
to develop an accurate understanding of impact, facilitate contextual lesson drawing, or consider 
implications for equity. 

A recurring critique of traditional impact evaluation is that it sheds little light on the program 
or policy mechanism through which change occurs (or does not occur). Theory-based evalua- 
tion has been mooted as a solution to this. The idea in a theory-based evaluation is to develop 
and test hypotheses that connect the policy intervention and its outcomes through explicit 
causal mechanism(s). Ilustratively, Martin Persson and Alpizar (2013) posit that pre-program 
compliance and selection bias influence the success of conditional cash transfer schemes and 
payment for ecosystem services; subsequently, they combine a multi-agent model with econo- 
metric analysis to show that selection bias can significantly reduce the desired impact. Similarly, 
Ssengooba, McPake, and Palmer (2012) combine a theory-based approach with a prospective 
case study design to assess a complex health system intervention in Uganda. 

In principle, the techniques and tools discussed here are available to any evaluator. However, 
given their almost exclusive focus on causality and their methodological sophistication, they are 
more likely to be employed in external or hybrid evaluation by academics specifically trained in 
these techniques rather than in internal evaluations often conducted by generalists and oriented 
toward practice and program improvement. 


Procedural Issues in Evaluation 


Thus, many of the more rigorous techniques, tool, and approaches discussed here might lend 
accuracy and credibility to an evaluation, but they do not necessarily translate into policy learn- 
ing and redesign. For the latter, relevant background information, the perceived expertise of the 
evaluator, and the consistency of the findings with prevalent beliefs within the organization also 
play a role (Yusa, Hynie, & Mitchell, 2016). Consequently, procedural instruments that influ- 
ence these also require consideration (especially in the case of a hybrid or internal evaluation). 

It is also the case that an internal evaluation can occur over a long time frame, something 
external evaluators find difficult to fund. In case of the United States Environmental Protec- 
tion Agency Superfund community involvement program, for example, the assessment was 
conducted in four phases over eight years (Charnley & Engelbert, 2005). The evaluation team 
focused on the role of public participation and the ability of the program to meet the partici- 
pants’ objectives by combining focus group discussions, telephone interviews, and written mail 
surveys. In such situations, institutionalizing contextual knowledge about the program or policy 
is a key requirement for the evaluation team. 

Governments can use procedural policy instruments to draw on external expertise for an 
‘internal’ evaluation. This might be accomplished, for example, by requiring audit committees 
to include an ‘independent?’ member. The Canadian Federal Accountability Act of 2006, 
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illustratively, mandated the creation of audit committees consisting of members of the public 
sector, academia, and the private sector (Shepherd, 2011). An alternative is the establishment of 
advisory commissions, in which well-known experts are enrolled to inform policymaking. The 
use of the findings of such commissions and audits, however, is dependent on the economic and 
political situation, the eventual rapport between the commission and the commissioning agency, 
and public mood (Clark & Trick, 2006). 

Further, meta-evaluation can also be a tool to support (evaluation) capacity development. 
Illustratively, Cousins et al. (2008) conduct a survey of internal evaluators in Canada to better 
understand organizational capacity, evaluation activity, and learning. Seppänen-Järvelä (2005) 
also highlights the importance of managerial, analytical, and political capacity to conduct 
process-oriented evaluation and use it for organizational development. The use of an evaluation 
capacity needs assessment tool to identify the organization’s strengths and weaknesses has also 
been proposed (El Hassar, Poth, Gokiert, & Bulut, 2021). 

Finally, the politics of evaluation can also influence its design and, consequently, the choice 
of technique deployed. Illustratively, this is highlighted by a study of an internal evaluation of an 
innovation policy framework in Ontario. In that study, Sharaput (2012) notes that the evalua- 
tion team selected simple indicators to demonstrate policy success rather than more sophisticated 
measures that might have shed light on whether the program had any effect and, if so, what. 
Apart from the absence of tools for causal attribution, a key challenge here was the normative 
context in which the agency had access to protected data of the policy target only during the 
course of the intervention. 


Application to the Case of Aid Effectiveness 


In this section, we illustrate the workings of the different aspects of external and internal evalu- 
ations mentioned earlier and their strengths and weaknesses using the case of the Asian Devel- 
opment Bank (ADB) and the evaluation of its multibillion-dollar multilateral development aid 
programs. 

The ADB is a multilateral financial institution that aims to catalyze socio-economic develop- 
ment in the Asia Pacific. Its projects and programs involve providing grants, loans, or technical 
assistance to more than 40 countries in the region in the following areas: agriculture, natural 
resources, and rural development; education; energy; finance; health; industry and trade; infor- 
mation and communication technology; public sector management; transport; and water and 
other urban infrastructure services (ADB, 2014, 2017a). It has a long track record of efforts to 
evaluate its efforts and has deployed many techniques to do so over the course of its history. 

As a large financial donor, the ADB is frequently scrutinized for its performance and prac- 
tices. To establish credibility and promote learning, the bank conducts various types of evalu- 
ations at the institutional level (for example, the Annual Evaluation Review and the Annual 
Report on Portfolio Performance), the strategy level (for example, the Country Assistance Per- 
formance Evaluation and the Country Partnership Strategy Final Review), and the project or 
program level (for example, the Project/Program Completion Reports [PCR] and the Technical 
Assistance Completion Report [TCR]). 


Internal Evaluations of the Asian Development Bank 


For projects and programs, the ADB has instituted two levels of internal evaluation: a self- 
evaluation by the respective project or program team and an evaluation by the Independent 
Evaluation Department (ADB, 2021). For example, every grant or loan project and program 
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is evaluated by the team responsible for project or program execution in its PCR. Similarly, 
every technical assistance project and program is evaluated by the responsible team in a TCR. 
Subsequently, a project or program may be evaluated by the Independent Evaluation Depart- 
ment. In the case of a grant or loan, this can take the form of a short Validation of Project/Pro- 
gram Completion Report (PVR) or a more detailed Project/Program Performance Evaluation 
Report (PPER). Meanwhile, in the case of technical assistance, this would be documented in a 
Technical Assistance Performance Evaluation (TPER). 

The evaluation itself is based on four criteria that span input, process, and output: effective- 
ness, efficiency, relevance, and sustainability. While effectiveness, efficiency, and sustainability 
evaluate the implementation of an intervention, relevance pertains to its design. An intervention 
is evaluated on a scale of 0 to 3 on each criterion, with 0 indicating lack of success and 3 indicat- 
ing success. Based on its performance on these criteria, it receives an overall rating: ‘unsuccess- 
ful’, ‘partly/less than successful’, ‘successful/generally successful’, or ‘highly successful’ (ADB, 
2006). For simplicity, a generally or highly successful project or program is considered successful 
by the institution (ADB, 2017b). 

Apart from the overall rating, the evaluation team also draws lessons from the experience. 
These lessons cover different aspects of the project or program, such as management, financing, 
approaches or methodologies, capacity development, policy reform, sector-specific issues, and 
cross-cutting issues. In principle, the lessons are expected to promote organizational learning 
and inform the design and implementation of future projects or programs. 

Using computational text analysis of nearly 1,000 project or program completion reports 
of the ADB, Goyal and Howlett (2019) examine the lessons mentioned in internal evaluations of 
the ADB. The authors find that these evaluations focused primarily on micro-level issues in the 
project or program, such as selection and scope, planning and execution, supervision, and the 
responsible team. Some meso-level issues, such as policy environment and sub-national and sub- 
sectoral determinants of project success, were also highlighted. In contrast, internal evaluations 
said little about macro-level issues — such as form of governance, macroeconomic policy, general 
standard of living — that might also affect project or program success. 


External Evaluation of Multilateral Development Aid 


Much of the early research on aid effectiveness was outcome based, focusing on its effect on 
variables related to development, such as economic growth. These evaluations resulted in mixed 
findings as some scholars showed that aid had a positive impact while others found that it had 
zero or a negative impact. Based on the recognition that context was important when inter- 
preting such differing evidence, more recent literature on aid effectiveness — and international 
development — has prioritized creating ‘micro-level evidence on what works and what does not. 

The case of multilateral development aid is no different. Increasingly, studies have ‘evalu- 
ated’ projects or programs of development banks to explain the causes of their success or failure 
rather than studying their long-term impact(s). However, as mentioned earlier, ‘external’ evalu- 
ators may not have access to the internal data of the organization. Consequently, such studies 
invariably use only publicly available data from internal evaluations — such as the economic rate 
of return or the success rating of the project or program — to shed light on project or program 
performance. 

Most of these studies rely on econometric techniques for deductive, hypothesis-driven research 
for such evaluation. Illustratively, in an early study on the topic, Isham and Kaufmann (1999) use 
projects’ ex-post economic rate of return, a measure of investment productivity, as an output to 
evaluate aid effectiveness at the World Bank through statistical analysis. They find that absence 
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of ‘distortion’ in exchange rate, macroeconomic pricing, and trade policies was associated with 
higher economic rates of return. Focusing on more immediate characteristics, Kilby (2000) uses 
econometric modeling to examine the effect of early project supervision, measured as time spent 
on the activity, on project performance. The author finds that more supervision was associated 
with higher likelihood of success, independent of the country or sector of operation. 

In the case of the African Development Bank, Mubila, Lufumpa, and Kayizzi-Mugerwa 
(2000) find that both micro-level characteristics — such as project size and sector — and 
macro-level characteristics — such as the rate of economic growth, inflation, and per capita 
income — determined project success. Denizer, Kaufmann, and Kraay (2013) evaluate micro- 
and macro-determinants of project and program success for over 6,000 projects or programs of 
the World Bank. The authors find that while institutional and policy environment was a key 
macro-level variable influencing project success, variation within the country was higher than 
variation between countries (indicating that micro- and meso-level characteristics played an 
important role). In their analysis, these micro-level characteristics included project size, project 
length, preparation, and project manager quality. In a comparison of the ADB and the World 
Bank, Bulman, Kolkma, and Kraay (2017) find that characteristics influencing aid effectiveness 
were largely similar between the two. 

As mentioned earlier, causal attribution is a key issue in external evaluations. To address this, 
Dollar and Levin (2005) employ the instrument variable technique in their econometric assess- 
ment. Using data from internal evaluations by the Operations Evaluation Department of the 
World Bank, the authors find that institutions — specifically, the presence of civil liberties and 
the rule of law — influenced project success on the whole, although their effect varied based on 
sector of operation. Feeny and Vuong (2017), too, employ the instrument variable technique 
in their analysis of project or program success at the ADB. The authors find that while at the 
macro-level, the rate of per capita economic growth and (low) level of democracy were related 
to project or program success, at the micro-level, the size of the intervention, the difference 
between anticipated and received funding, and the nature of the intervention (project or pro- 
gram) influenced outcomes. 

Using a different research design and method, Saguin (2017) evaluates the influence of criti- 
cal capacities on the success of administrative reform based on data from internal evaluations 
at the ADB. The author examines 20 PVRs on ‘public sector management’ using qualitative 
content analysis and complete-linkage clustering to find that the relationships among context- 
driven planning, coordination risk assessment, institutional support, instrumental political sup- 
port, and multi-stakeholder ownership influenced the success of administrative reform. 

Earlier, we identified theory-driven evaluation as one approach for identifying the mecha- 
nisms of the program or policy. In an example of such an approach in the case of aid effective- 
ness, Wane (2004) hypothesizes that the accountability and capacity of the recipient government 
were important in shaping project or program outcomes as they influenced their choice of proj- 
ect or program. Specifically, the author shows that governments with high accountability and 
high capacity were more likely to accept only well-designed aid projects than governments with 
low accountability or capacity. Such an evaluation can shed light on the likely causal mechanism 
at play and enable lesson drawing for enhancing effectiveness. 

Further, the turn toward ex-ante evaluation is also observed in the case of external evalua- 
tions of aid effectiveness. Geli, Kraay, and Nobakht (2014), for example, examine the predictive 
performance of various models of project success using data from the World Bank. Arguing that 
their model is more accurate at predicting success than self-evaluations during implementation 
by the project team, the authors use their predictive analysis for ex-ante evaluation of ongoing 
projects or programs of the World Bank. 
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While most studies focus on output- or outcome-based assessment, process-based evalu- 
ation has also been conducted. In one type of process evaluation, Buntaine (2011) examines 
whether and how prior environmental performance of a borrower influences aid allocation 
decisions of the ADB. The author used a binary regression model to analyze the relationship of 
aid allocation — based on data from the ADB — to an indicator of environmental performance, 
constructed with a Bayesian updating model. 

In their comparison of these external evaluations with internal evaluations of the ADB, 
Goyal and Howlett (2019) find that external evaluations complemented internal evaluations by 
highlighting more meso- and macro-level characteristics associated with program success. At 
the meso-level, for example, external evaluations indicated that sub-sectoral and sub-national 
variation in program success might be related to the features of the institutional and policy envi- 
ronment, such as civil liberties and the rule of law. Similarly, at the macro level, external evalu- 
ations found a strong association of program success with variables such as the rate of economic 
growth, civil liberties, and political freedom. 


Conclusion 


In this chapter, we showed how the approaches, tools, and procedural issues in evaluation vary 
depending on the location of the evaluator (i.e., internal or external to the organization being 
evaluated). The common approaches for policy evaluation include cost-benefit analysis, multi- 
criteria decision analysis, life cycle assessment, and impact evaluation. Increasingly, the first three 
are also used for ex-ante assessment as well as ex post in order to help select among alternatives 
and inform policy design. Impact evaluation, while also used ex ante, has witnessed method- 
ological advances and practical interest especially for ex-post causal attribution in the recent 
past, for instance, as seen by the growing trend of experimental research in the social sciences. 
Apart from the selection of relevant approaches and appropriate tools, evaluations also involve 
procedural issues, such as its institutionalization, a plausible tension between the use of external 
expertise and the incorporation of the findings, the assessment and development of evaluation 
capacity, and the politics of evaluation. 

Although these choices could influence both internal and external evaluators equally, in 
practice, the response is likely to vary depending on the type of evaluator (and evaluation). 
External evaluators such as academics typically possess higher analytical sophistication and more 
autonomy in designing the evaluation than do internal evaluators. Yet they are also often con- 
strained by data availability and lack of tacit knowledge about the program. Consequently, they 
are more likely to opt for a sophisticated design and advanced methodological toolkit but a 
comparatively narrow focus that permits generalizability. In contrast, internal evaluators usually 
have more information and contextual knowledge about the intervention but stronger financial 
or professional ties with the organization, lesser analytical capacity, and more concern regarding 
the use of the findings and the politics of evaluation. Therefore, they are likely to prefer more 
holistic designs, simpler methods, and an institutionalized approach but with less attention to 
aspects such as causal attribution. 

These differences between internal and external evaluations were illustrated in the case of the 
evaluation of aid effectiveness. Here, we observed that the ADB has a standardized approach that 
combines self-evaluation by the project team with evaluation by an independent department 
for their projects and programs, regardless of whether they are grant based, loan based, or based 
on technical assistance. Further, the institution adopts broad criteria to measure performance — 
spanning input, process, and output — and emphasizes drawing lessons about different aspects 
of the intervention. Internal evaluations, however, tend to focus on micro- and meso-level 
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issues and not more macro- or meta-institutional ones. External evaluations of interventions by 
the ADB or other multilateral development institutions, on the other hand, usually combine 
‘internal’ data with other socio-economic or political variables to analyze the more macro-level 
determinants on program success through more methods that allow generalization across several 
sectors, countries, and even institutions. As a result, internal and external evaluations in practice 
can be seen as often complementing each other: as internal evaluations shed light on micro- and 
meso-level characteristics of program success while external evaluations create insights regarding 
more meso- and macro-level characteristics. 

Hybrid, or multi-level, evaluation has thus been proposed as an approach that can combine 
the strengths of internal and external evaluations while overcoming their limitations. Seen from 
the perspective of tools of governance, this is akin to adopting a policy mixes approach to the 
design of evaluation, which could be promising so long as the tools in the mix are synergistic 
and complement one another, as would be the case for any other such combination or mix of 
tools. 
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ADMINISTRATIVE PROCEDURE 
AS A TOOL FOR PARTICIPATORY 
AND EFFICIENT PUBLIC 
GOVERNANCE 


Polonca Kovaé 


Administrative procedures and their outcome-oriented codification and consistent implementa- 
tion are among the most salient aspects of sound public governance. Most public policies are legally 
determined in policy- or rule-making and single-case decision-making procedures that comply 
with fundamental principles, such as the rule of law, accountability, transparency, inclusion, and 
effective public service. This chapter addresses administrative procedures as a dialogue and evalua- 
tion tool of democratic public affairs, regardless of the prevailing governance model — e.g., (neo-) 
Weberian or (post-)New Public Management — and irrespective of their codification mode. As 
regards the latter, some countries have adopted a general statute (an APA) while others prefer 
detailed sector-specific laws or codes of conduct. The analysis shows that a systemic approach 
to administrative procedures can contribute to a more agile public administration, particularly 
through the proportional protection of the public interest and citizens’ rights under various types 
of administrative acts. Thus, worldwide, procedures are becoming an increasingly important part 
of efficient and participatory public administration. 


Introduction 


The function of administrative procedure as an evaluation tool in public policymaking is also 
important, be it in the issuance of administrative regulations or single-case decisions (Hofmann 
et al. 2014; Kovač 2017). If public policymaking is understood as a system for analyzing the situ- 
ation in society, designing and selecting alternatives to improve the situation, implementing the 
selected solutions, and again analyzing the situation, one can see that administrative processes are 
the key components of the plan-do-act-check loop. For administrative regulations, this applies 
in the stage between the selection of the goals pursued by parliaments and governments and 
their operationalization, which falls within the domain of the public administration as the de 
facto legislature, and drafting laws and acts for their implementation. 

Administrative rule-making, therefore, entails institutional public governance, which should 
include, inter alia, ex-ante regulatory impact analysis (RIA) and public consultation while single- 
case administrative acts or procedures can be considered instrumental public governance: i.e., 
the implementation of the regulations in practice. The latter procedures reflect — through the 
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disputability of relations with legal remedies and their necessary enforcement — whether the 
objectives set in the laws and implementing regulations are properly implemented. If not, at the 
aggregate level, these single-case issues point to the need to launch a new regulatory cycle and 
adopt better regulations. Therefore, single-case administrative procedures in particular are impor- 
tant to complete the regulatory feedback loop as they — intentionally or unintentionally — imply 
the ex-post evaluation of public policies. 

The term administrative procedure can cover different phenomena or relations, largely depend- 
ing on the administrative and legal system in place. The differences rest on the general social, 
political, and economic development of the individual country or administrative tradition, as 
well as on the specific relations between the branches of power as public policymakers and the 
types of procedures and acts resulting therefrom. However, public relations strive for good public 
governance that encompasses various social and legal interests and takes them into account in the 
formulation and implementation of effective public policies while pursuing the public interest 
(see Venice Commission 2011; Bevir 2011; Bouckaert and Jann 2020). 

This chapter addresses different types of administrative procedures, depending on the powers 
attributed to the public administration as part of the executive branch of power. It distinguishes 
in particular between two types of procedures: (1) administrative procedures in sensu lato, result- 
ing in the creation of implementing regulations — mainly government decrees or ministerial 
rules — specifying in more detail the enforcement of legal rights and obligations relating to public 
policies and (2) administrative procedures as applied by most national systems: i.e., single-case 
administrative decision-making to determine rights and obligations in relations between author- 
ities and individuals. The latter cover a multitude of administrative matters, from collecting taxes 
and granting social benefits to inspections and issuing building permits, public documents, visas, 
etc. (cf. Hofmann et al. 2014; Auby 2014; Harlow and Rawlings 2014). 

There are many differences in national or regional practices as regards the principles and rules 
governing such procedures. However, thanks to the transfer of good practices, even relatively 
opposite traditions — such as the American and German ones, and even more so in Europe — 
seem to converge over time (Ktinnecke 2007; Kovač and Sever 2014). Convergence is thus 
observed in the codification of administrative procedures with efforts to regulate the administra- 
tive relations between authorities and individuals holistically. Proof thereof is some new, hybrid 
procedures that cannot be classified into the two basic types (general and single-case procedures 
and acts). It is even more important to understand that administrative procedure — with its tradi- 
tional codification of general principles — represents only one or several stages of the public pol- 
icy or regulatory cycle, as administrative rule-making operationalizes institutionally determined 
public policy objectives. Single-case decision-making — as instrumental public governance — also 
reflects whether and to what extent laws and parliamentary policies are effective in practice (see 
Kovač 2017). Moreover, convergence presents an advantage over the otherwise different systems 
in place in individual countries. As administrative procedures also regulate cross-border situa- 
tions, the common values and principles of public governance are likely to produce comparable 
rights and restrictions for the parties. In such regard, an important role is also played by global 
organizations such as the OECD and the Council of Europe, which promote good or sound 
public governance as an element of sustainable social development (OECD 2011, 2017; Venice 
Commission 2011). 

In view of these common baselines and differences, the research question is which system (of 
codification) of administrative procedure best serves to ensure good public governance, consid- 
ering the established instruments for the operationalization of its principles. Another question 
is whether it is possible to establish common principles and instruments for all types of proce- 
dures in public administration that would harmonize the work of the administration regardless 
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of whether the result of the administrative procedure is administrative regulations, single-case 
administrative acts (decisions), or hybrid acts (e.g., administrative contracts, single-case guar- 
antee acts for future situations), considering that they all authoritatively address citizens and 
companies. The principles guiding the procedure are, in fact, the result of its functions while 
procedure — as an institution — embodies the status quo of public law at any moment. Con- 
sequently, (any) administrative procedure reflects the given understanding of the citizen-state 
relation (Barnes in Rose-Ackerman and Lindseth 2010: 337). In terms of participation, for 
example, the minimum common procedural standards are the involvement and familiarization 
of the directly affected parties with the decision. Therefore, participation must take precedence 
over efficiency in the narrow sense (Kovac 2018). In this respect, democratic procedures are not 
only a political instrument legitimizing the conduct of people’s representatives and thus, indi- 
rectly, of public administration but also an element of direct decision-making or at least citizens’ 
participation in public affairs. 

The present chapter first defines the concept, models, and principles of good governance 
and the role of administrative procedures. This is followed by an analysis of the main types of 
administrative procedures and their specific rules. The analysis is upgraded with a discussion of 
which system is more optimal, considering the existing state of the instruments of good gover- 
nance (GG) and how administrative procedures should therefore be (generally) codified in the 
future. In such regard, I draw from the multifunctionality of administrative procedure as a tool 
of dialogue between public authorities and private parties that should result in the proportional 
recognition of public and private legal benefits and a regulatory feedback loop for the formula- 
tion and implementation of public policies. In this sense, administrative procedures or various 
other processes serve as an instrument to balance social values and interests while also represent- 
ing an evaluation tool mirroring the (in)efficiency of public policies and regulations in real-life 
events. This is especially important in times of global crises, such as financial and migration crises 
or pandemics, which require public administration to be agile, efficient, and still democratic. 


Administrative Procedures and Their Codification 
Within Public Governance 


Traditionally, there are some key differences among individual societies. These relate, for exam- 
ple, to the type of government and bearers of administrative and other authority or to the rela- 
tions between them (for more, see Karpen and Xanthaki 2017; Raadschelders 2011; Schuppert 
2000). This leads to widely varying organizational structures of administrative bodies: e.g., sin- 
gle, two-tier, or multiple-tier authorities at the national, regional, and local levels. Multi-level 
governance — e.g., in the European Union (EU) or federal states — and the disciplinary under- 
standing of administrative procedures also play an important role. In the Central European con- 
text, administrative law is deemed to be a part of the domain of public law relations, in which 
the administrative body is per definitionem superior to the party, while in Anglo-Saxon systems, 
individual relations and disputes with authorities are, in principle, regulated more flexibly and 
amicably, even in adapted litigation proceedings. Moreover, the differences between systems may 
be due to the democratic tradition, with new democracies — such as those in Eastern Europe — 
presenting an implementation gap between strategies/regulations and practice (Kovac and Bileisis 
2017: 11ff). Other factors of differentiation include the scope and content of administrative regu- 
lation, the hierarchy of acts, and regulatory density (1.e., the scope and detailedness of regulations) 
(Karpen and Xanthaki 2020: 8ff). 

However, despite their differences, most countries feature a more or less systemic codification 
in the form of a (General) Administrative Procedure Act (APA) (see Ziller in Peters and Pierre 
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2005: 260ff Auby 2014: 4) aiming at equality and lawfulness and the legal determination of 
authoritative interferences with the rights and obligations of individuals. Spain, for example, 
adopted its APA as early as 1889, Austria in 1925, the United States in 1946, Germany in 1976, 
Japan in 1993, Slovenia in 1999, Estonia in 2001, Finland in 2003, and France in 2015. Some 
APAs provide common rules for different types of administrative procedures and acts while most 
of them — especially in the European context — only cover single-case decision-making. Fur- 
thermore, some APAs set out common principles for all administrative procedures regardless of 
the specific field — e.g., taxes, the environment and construction, and home and social affairs — 
which sector-specific regulations may openly regulate otherwise. In other cases, this issue is sub- 
ject to special requirements for the equal protection of rights, with deviations only in the event 
of reasonable grounds for differentiation or even with no differentiation at all. The latter applies 
in most European countries for specific constitutional guarantees (Hoffmann-Riem et al. 2012: 
510ff cf. McCubbins in Coglianese and Kagan 2007: 19). These are required, for example, by 
the rights of defense such as access to one’s own files and the right to be heard, devolution and 
the suspensive nature of an appeal, and the judicial review of the legality of administrative acts 
for effective legal protection against the administration. Another difference concerns the level of 
detailedness or the definition of principles and fundamental rights, as well as the basic or detailed 
operational rules. Therefore, certain APAs have just over 10 articles (e.g., the US APA only 16), 
some have between 30 and 100 articles, while others comprise well over 300 articles and numer- 
ous additional specific and implementing rules. 

In any case, from the viewpoint of political science, as is characteristic in US theory (see 
Craig in Peters and Pierre 2005; Coglianese and Kagan 2007; Kovač and Sever 2014), admin- 
istrative procedures are examined as a consequence of or a tool for exercising state authority. 
This is the case both in terms of adopting general rules at the level of the administration and 
in terms of single-case decisions or administrative procedures as key components of administra- 
tive reforms. On the other hand, most European systems are more conservative and understand 
administrative procedures in a narrower sense (Auby 2014: 3ff). 

Two groups of procedures and their respective codifications are usually distinguished: (1) 
rule-making or the issuance of administrative regulations as policy-based decisions with general 
effect or procedural arrangements (i.e., the public policy cycle) (German: Gestaltungsverfahren) 
and (2) the issuance of individual and, as a rule, concrete administrative acts (i.e., single-case 
decision-making) (German: Verwaltungsverfahren) (cf. Hofmann et al. 2014). In view of this, the 
regulation of administrative procedure is indeed important for enforcing the goals of the party in 
power. In fact, politics is also conducted by defining the procedural rules: e.g., a priori exclud- 
ing the holders of certain interests and thus consciously steering social interactions to match 
the values of those in power. The regulation of administrative procedure is therefore necessarily 
bound with politics. The 1946 US APA (see McCubbins in Coglianese and Kagan 2007: 3ff), 
for example, aims at “fairness in administrative operation” and “the effectuation of the declared 
policies of the Congress.” Therefore, the role of administrative procedure is defined and changes 
depending on the role of the state in the society, the tasks of the authorities, the consequent 
understanding of the principle of the separation of powers between the legislature and the 
executive, and the constitutional arrangements determined thereby. The latter also depends on 
the selected governance model, which influences the definition of administrative procedures or 
their basic principles in either specific or general matters. 

In order to assess which solutions are better, it is necessary to further analyze the basic models 
of public governance as different administrative systems different principles and codifications of 
administrative procedure come to the fore, depending on the administrative tradition. Some sys- 
tems favor a more legally defined approach, as is typical of the continental European Rechtsstaat 
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or Napoleonic tradition, while the Anglo-Saxon tradition relies on principle-based regulation 
and flexible tools, albeit with a public interest orientation. Certain systems are more authorita- 
tive (e.g., the German system) while others are participatory and transparent (e.g., the Scandi- 
navian systems). Compensating for the (otherwise) superiority of the public interest inherent in 
administrative relations are the party’s right to be heard and, generally, participation in public 
policymaking. However, in administrative relations, equality between participants still implies an 
a priori asymmetry compared to civil proceedings in terms of both the possession of information 
and decision-making power. Authorities may indeed foster dialogue with the parties and strive 
to give them a say in the final decision, but this may well be just “window-dressing” (Pitschas 
and Walther 2008: 99ff). Such an approach to initiating and conducting administrative proce- 
dures reduces the need for inquisitorialness (Ktinnecke 2007: 46; Auby 2014: 14) and leads to 
better-enforced public policies (Tyler 2006: 5). 

Among the models influenced by the understanding of administrative procedures, three main 
models are worth highlighting (cf. the Venice Commission 2011; Mathis 2014; Raadschelders 
and Vigoda-Gadot 2015; Bouckaert and Jann 2020). When determining their public gover- 
nance through rules and holders of authority, countries gather into clusters leaning toward one 
of the three models, although there is no ideal or pure approach in practice. That is why new, 
mixed concepts are being developed, such as the New Weberian State, New Public Governance, 
and Digital Era Governance. 

For the purpose of this chapter, we will limit ourselves to only presenting the basic three 
models. They are listed in chronological order: 


Neo-Weberian administration with a rather formal understanding of the rule of law and 
the superiority of administrators in relation to citizens, as within the Rechtsstaat tradition; 
administrative procedures are only perceived as single-case unilateral decision-making; 

(Post-) New Public Management, which advocates public administration as a service simi- 
lar to private services, supports the involvement of citizens in administrative procedures 
as equal counterparts to the authorities, and strives for alternative dispute resolution 
(ADR), agreement, and a reduction in the administrative burden; and 

Good (public) governance, encompassing several fundamental principles at all levels of public 
administration, from lawfulness, participation, and accountability to responsiveness, transpar- 
ency, and efficiency; what is important for the codification of administrative procedures is a 
holistic view and a uniform and more abstract regulation by which different principles are 
applied at different stages and in different degrees, depending on the conflict of interests. 


Clearly, in terms of integrity and principled orientation, the most comprehensive and 
advanced is the third approach. This can also be attributed to the weaknesses of the other two — 
e.g., the Weberian formalism of administrative decision-making or managerial politicization — or 
the erosion of public interest protection should participation and efficiency prevail over legality 
(Mathis 2014; Kovac and Bileisis 2017: 471ff). Dysfunctions also arise from GG (e.g., an outflow 
of democratic accountability due to intensively emphasized networking), but there seem to be 
fewer of these as a systemic context is key to the implementation of GG principles (OECD 2011; 
Bevir 2011). 

Modern public governance has, in fact, recognized the need for inclusive approaches that 
take into account several stakeholders. Accordingly, in the 1980s, traditional closed-ended gov- 
ernance models based on Weber’s theoretical assumptions were gradually replaced by alternative 
governance models designed to make better use of the administration’s potential and to better 
meet the challenges of modern society — such as New Public Management — although there 
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seems to be some disregard for the participation of stakeholders as active citizens. Such develop- 
ment stems from the demands for a reduction in public expenditure and the greater involvement 
of external recipients of public services and civil servants, but only on a declarative level. The 
countries of continental Europe have therefore developed in particular the neo-Weberian model 
and other models of governance, with each new model presenting an improvement over the 
previous ones. With regard to such, systemic holism is becoming increasingly noticeable, mainly 
expressed in the participation of a wide range of stakeholders. 

Furthermore, one should not ignore the doctrine of good administration (GA) developed 
within GG (see Venice Commission 2011; Harlow and Rawlings 2014: 87ff Galetta et al. 
2015). GA comprises a set of fundamental principles and rights that — given the superior- 
ity of administrative authority in administrative matters due to the a priori protection of the 
public interest — are crucial for the democratic protection of the rights of individual parties in 
administrative procedures. These are principles that are more or less enforceable than individual 
rights, such as legitimate expectations, equal treatment, the duty of care, a fair hearing, access to 
information, data protection, timeliness, and an effective remedy (for more, see Hofmann and 
Mihaescu 2013). 

What is key in such regard is the general codification of administrative procedure typical of 
most countries in the world and of the EU (see Auby 2014; Harlow and Rawlings 2014) as it 
introduces common fundamental principles and relevant rights, regardless of the fragmentation 
of administrative areas and specific procedures. It is important that GA, as a part of GG, defines 
the main value guidelines in the relation between the authority and the addressees of authoritative 
acts. In doing so, the concept seeks to find a balance between public and private interests, taking 
into account the stage of the procedure or the definiteness of the legal status of the parties, the 
degree of conflict, the scope of validity, and thus the extent of the persons affected. 

Despite the different political and legal systems, all social systems globally present similar 
or even convergently evolving principles and rules of administrative procedures and relations 
(Galetta et al. 2015; Kiinnecke 2007). Globalization, digitalization, delegation, and similar pro- 
cesses have contributed to the study and improvement of good and bad national or regional 
practices and their transfer from one system to another. In this sense, there is at least conver- 
gence, if not harmonization, in the field of administrative procedure, as shown in particular in 
the EU with the drafting of a single regulation as a “European APA.” This act is a collection 
of good national practices and the realization of the concept of good administration (Hofmann 
et al. 2014). It was initially intended to serve as an umbrella framework for all administrative 
procedures regardless of the type or holder of authority in the EU and Member States but was 
later limited to European institutions in single-case administrative decision-making (Harlow 
and Rawlings 2014: 332ff). But despite the positive expert assessments of the two resolutions 
adopted by the European Parliament in 2013 and 2016 and the public support expressed in 
public consultations, there are still reservations on the part of, for example, the European Com- 
mission. The fact that the resolution of June 9, 2016, for an open, efficient, and independent 
European Union administration (2016/2610(RSP); European Parliament 2016) is still in its draft 
stage clearly shows that there is a long way to go from theoretically conceived ideals to an actu- 
ally adopted and implemented regulation in practice. 


The Multi-Functionality of Various Administrative 
Procedures and GG Principles 


Traditionally, administrative procedure has served as a tool for balancing collisions between the 
public interest and the rights of parties. In such context, considering the significance of the 
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protection of the constitutional safeguards of parties against authorities, procedural and constitu- 
tional law consider certain de jure procedural rights to be a constituent part of the subject mat- 
ter of procedure. The key function of administrative procedural rules is therefore the balanced 
protection of an individual party in the procedure — the public interest must indeed prevail, but 
not absolutely. This also applies when administrative procedure is understood more broadly: 
i.e., including the issuance of administrative regulations. As their scope is greater, there is also a 
greater need to balance several interests. Administrative procedure can thus serve as a basic tool 
to ensure lawfulness and democracy (Peters and Pierre 2005: 261ff). 

Procedure or process could thus also be defined as a structured process of choosing between 
several possible alternatives based on the acquisition and processing of information (Hofmann 
et al. 2014). This definition essentially applies to the adoption of both administrative regulations 
and single-case administrative procedures, which some authors express through the distinction 
between the broader process and narrower — especially single-case — procedures (see Pitschas 
and Walther 2008; Harlow and Rawlings 2014: 2ff, specifically on administrative regulations in 
the EU, ibidem: 93-118). In a broader sense, this could also include courts. Given the intensity 
with which they oversee administrative activities, courts in Central Europe can act as an active 
formalist body ensuring judicialized administration. In contrast, Anglo-Saxon courts and judges 
present a high degree of activism and low degree of formalism, thus acting as drivers of fairness 
redefined (according to Harlow and Rawlings 1997: 516; Ziller in Peters and Pierre 2005: 173). 
Thus, in managing public affairs in the modern state, one must take into account the correlation 
of all the functions of administration as one of the branches of power, as well as the globalization 
or externalization of administrative matters across national borders and the efficiency and ratio- 
nality of pursuing the common good as a “patchwork” of area-specific public interests (Pitschas 
and Walther 2008: 11-31). 

When a procedure is initiated, its objective is not yet clearly defined. In fact, it is steered 
throughout the procedure by a number of unpredictable interactions between the participants 
and consequent procedural actions (Schuppert 2000: 772ff Hoffmann-Riem et al. 2012: 523ff). 
The purpose of the procedure is therefore to reduce uncertainty as to its objective, whereby a 
certain degree of uncertainty in the procedure is unavoidable, considering the creative contribu- 
tion of an official in applying a general norm or rule to a specific factual situation. When, how- 
ever, a procedure is perceived even as only single-case decision-making, its formal lawfulness 
ensures predictability and thus legal certainty, the transparency of public tasks, and the sensitivity 
of authorities to legitimate expectations, drawing from the concept of personal dignity (Harlow 
and Rawlings 1997: 497; cf. Tyler 2006). 

Procedural rules thus ensure substantively correct or lawful decisions while protecting certain 
fundamental human rights. This applies in particular to procedural institutions related to the 
rights of participation and defense (e.g., the right to be heard or to the issuance of a decision 
without undue delay). These procedural rules also express other functions of the procedure 
(e.g., the expression of authority, the acceptability of decisions, the counterbalance of judi- 
cial protection, economic and investment development, political efficiency, and the stability of 
democracy in public governance. According to some authors, only the procedural elaboration 
of a substantive right enables its implementation, legal remedies included (Schuppert 2000: 810; 
cf. Hofmann and Mihaescu 2013). 

(Administrative) procedure thus serves the objective it pursues, in the sense of the realization 
of a substantive right as the subject of the procedure. At the same time, it is understood not only 
as a tool but also as a goal of the procedure per se. Thus, as public administration evolves, the role 
of administrative processes and their legal regulation increases over time, especially in the frame- 
work of GG (OECD 2017). Likewise, the importance of procedural rules grows, particularly 
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when the subject matter of public policies is hard to determine given the increasingly complex 
relations in society or the technical nature of the subject of regulation (Peters and Pierre 2005: 
284; Rose-Ackerman and Lindseth 2010: 342). 

Procedure is the fundamental framework for pursuing justice by balancing legitimate inter- 
ests, be they in the Anglo-Saxon, German, Francophone, or any other tradition. However, 
given the presence of authoritative interventions, such procedure must be at least partially regu- 
lated by law as procedural rules often determine its substantive outcome. The codification of 
administrative procedures or administrative procedural law is therefore considered to be the core 
of administrative relations. Of course, this applies as long as procedures and, in particular, their 
implementation under administrative procedural law are efficient. Yet efficiency must be under- 
stood as the right ratio between the necessary measures to protect the public interest and the 
rights of the parties. Care must be taken so that the importance of the procedure and especially 
its legal regulation do not reach the other extreme where form becomes an end in itself. The 
degree of legal regulation of administrative relations and the authoritativeness of the cogent law 
therefore depends on the conflictuality of the relations and the severity of the interference with 
the legal position of the parties. Equally detailed regulation in all cases is thus neither necessary 
nor useful (Harlow and Rawlings 1997: 504; Künnecke 2007: 46ff Kovač 2018). 

Not all principles and procedural guarantees have the same weight. The importance of 
administrative procedural institutions is inevitably related to the right that is the subject of the 
procedure (for more, see Hoffmann-Riem et al. 2012, who, for example, highlight situation- 
based rules or composite or staged procedures: e.g., in relation to public information or asylum 
matters). Depending on the nature of the case, procedures differ also in terms of the manner 
of their regulation and not just in terms of the content or type of the rights. Thus, as a rule, 
the procedure for issuing general acts is less defined while the procedure for issuing single-case 
administrative acts is determined in more detail and in an objective, politically neutral manner. 
Therefore, it is first necessary to define the type of the relation to be regulated by law in order 
to be able to determine the type of procedure to be applied to pursue legally protected interests 
in a particular relation. 

The main difference concerning the types of administrative processes lies in the level of the 
determination of the interests involved and, consequently, the tangibility of the rights, which 
increases with greater determination of the participants in the procedure. The procedures in 
which administrators issue general and individual legal acts in any case comprise a considerable 
scope of administration competences, especially in more Weberian- and Rechtsstaat-oriented 
countries. However, the two functions are attributed theoretically to a “lower” level of public 
governance (rowing, instrumentalism) as opposed to “higher” general decision-making (steer- 
ing, institutionalism) while, in practice, administrators most often have a more influential say 
(Kovač 2017; cf. Raadschelders and Vigoda-Gadot 2015). 

Due to the redefinition or expansion of the concept of administrative procedure through the 
GG doctrine — which in ever more countries (no longer) entails only authoritative decision- 
making on the individual rights, legal interests, and obligations of the parties based on general 
rules and concrete facts — the essential functions of procedure are changing. In this context, 
partnerships are established even between participants in traditionally authoritative administra- 
tive relations, such as in Central Europe (see Schuppert in Bevir 2011; Künnecke 2007; Kovač 
2018). A participatory administration is one of the key building blocks of modern society, 
enabling citizens, companies, non-governmental organizations, and other parties to access infor- 
mation and to amicably reconcile their interests. This derives from the simultaneous instrumental 
significance of the procedure for the realization of the goal of the procedure — i.e., the exercise of 
a substantive right or legal benefit or the performance of an obligation in the light of the public 
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interest — and from the role of procedural institutions recognizing the parties as subjects and not 
merely as the objects of procedure. With growing transparency and, consequently, participation 
in regulatory decisions, the credibility of regulatory responses and the level of public trust in the 
authorities also increase (Schuppert 2000: 790; Tyler 2006: 16; Karpen and Xanthaki 2020: 73). 


Discussion and Recommendations Regarding Future General 
Procedural Codification 


A comparison of current regulations suggests that there are several political and legal instru- 
ments in both rule-making and single-case administrative procedures aimed at ensuring their 
multifunctionality at different stages, in accordance with the principles of GG. However, it can 
also be observed that some procedural stages are not covered, although there are mechanisms in 
place for that very stage and principle. This means that a systemic approach to “transplanting” or 
extending these methods to all administrative processes could significantly increase the efficiency 
of public administration (see, for example, Hofmann et al. 2014; and Kovač 2017). 

What is particularly interesting in such regard is that a larger number of well-developed 
institutions can be detected in single-case administrative decision-making than in the issuance 
of administrative regulations. This is a rather unique paradox, as the latter address more people, 
have a wider scope, and are thus more important for the exercise of authority. There are mul- 
tiple reasons for such, mostly stemming from the tradition of the development of administrative 
processes and the gradual transfer of legislative functions to the executive level. The delegation 
of rule-making to ministries or other administrative bodies is a result of the complexity and 
technical standardization of modern society (Barnes in Rose-Ackerman and Lindseth 2010: 
336ff; Karpen and Xanthaki 2020: 237). The latter requires noticeably faster responses to real- 
life situations (e.g., in crises such as the COVID-19 pandemic), to which legislative procedures 
in parliament are not adapted. This, however, does not speak against transposing the positive 
mechanisms of traditional single-case administrative procedures into rule-making. 

Looking at some examples of GG instruments, one can see that in rule-making, the principle 
of the rule of law and thus the accountability of decision-makers are reflected in the necessary 
compliance of lower acts with higher ones, especially laws. This is usually checked by national 
constitutional courts in years-long procedures, whereas accountability is more political than 
legal. In single-case decision-making, on the other hand, material (substantive) and formal (pro- 
cedural) legality are defined in more detail and enforced more rapidly and more easily, thanks 
to legal remedies (especially administrative appeals) and judicial review, along with responsibility 
for, for example, compensation or even the criminal liability of the authorities breaking the rules 
(Auby 2014: 24ff cf. Hofmann et al. 2014). 

The principles of equity and inclusion, consensus orientation, and, in particular, participation 
are expressed in rule-making through public consultation and engagement mechanisms while in 
single-case procedures, they are expressed through the right to be heard. In single-case proce- 
dures, the latter is again defined in much more detail and thus enforceable already in administra- 
tive appeal and further court proceedings (for both the German and Anglo-Saxon systems, see 
Kiinnecke 2007; Bevir 2011; Hofmann and Mihaescu 2013; Kovač and Sever 2014), based on 
the provisions of national APAs on the entitlement to such right, on the manner and time of its 
enforcement, and on violations thereof as substantial procedural errors and a direct reason for 
legal protection and the reopening of proceedings (Ktinnecke 2007: 138). However, a distinc- 
tion must be made between the participation of the parties (German: Beteiligung) in single-case 
administrative matters and broader participation (German: Partizipation) as co-decision-making 
with the rule-maker (Hoffmann-Riem et al. 2012: 699ff). 
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In different legal systems, the principles of transparency and responsiveness, and partly even 
participation, are enforced in rule-making mainly through a notice-and-comment mechanism, 
more by guidelines than actual rules (e.g., the deadline for comments on regulations is “as a rule” 
30 days, with numerous exceptions and no legal consequences for a violation). In single-case 
decision-making, these principles are expressed through individual types of procedural actions 
and both substantive and procedural rights ensuring public control and subject to judicial review, 
such as public hearings, access to files, access to public information (for more, by country profile, 
see Auby et al. 2014 and, for the EU, Harlow and Rawlings 2014; Galetta et al. 2015). 

Finally, the principles of effectiveness and efficiency are expressed in rule-making — in par- 
ticular in RIA — as an ex-ante analysis of the anticipated impact of regulations that is intended to 
provide evidence-based decision-making and evaluation in the regulatory feedback loop (Naun- 
dorf and Radaelli in Karpen and Xanthaki 2017: 187ff). At this stage, concern for reducing red 
tape is also important (Kovač 2017). In most systems, however, only the notice-and-comment 
procedure applies in rule-making while in single-case decision-making, decisions are made in 
compliance with APAs and the institutions determined thereby. These are, for example, rules 
on the deadlines and duration of the procedure or the consequences of administrative silence, 
as well as more modern approaches to reconciling interests, such as ADR (Dragos et al. 2014, 
2020; Pitschas and Walther 2008). 

Nevertheless, if any of the fundamental GG principles are not or are only partially imple- 
mented, at least the directly related principles — e.g., responsiveness and transparency — are 
undermined and, along with them, also accountability, lawfulness, and people’s trust in 
the authorities. Therefore, the problem must be solved holistically. In view thereof, third- 
generation administrative procedures are evolving that apply the administrative method to the 
policy cycle and thus emphasize the dialogue between (administrative) authorities and indi- 
vidual parties (Barnes in Rose-Ackerman and Lindseth 2010: 350ff). But in most administra- 
tive systems around the world, rule-making and single-case decision-making are still perceived 
as separate procedures, both theoretically and in terms of legal protection, despite sharing 
the same principles of democratic authority (McCubbins in Coglianese and Kagan 2007: 19; 
more in Bouckaert and Jann 2020). In such context, particular attention should be paid to the 
growing importance and scope of administrative relations. Administrative regulations, which 
in many countries number more than 20 times the number of laws (namely, 95% of all regula- 
tions are administrative and only 5% are parliamentary laws) directly affect the lives of everyone 
in society, which increases the importance of meta-regulation, especially for the legitimacy of 
the modern state (Karpen and Xanthaki 2017, 2020). Participatory governance in this sense 
bridges the democratic deficit typical of bureaucracy, which is not a democratically elected 
structure of authority. Modern society thus strives to include legitimacy within (administrative) 
lawfulness through the connection of administrative authorities and citizens or companies as 
addressees of administrative decisions. 

In adopting administrative regulations and issuing administrative decisions, in addition to for- 
mal principles and mechanisms common to both types of acts, participatory governance should 
apply to the pre-stage or informal part of administrative rule-making and single-case decision- 
making. Formulating and expressing opinions and preferences as to possible alternatives should 
be based on a system that is able to identify the best decisions based on the aggregation or (high- 
est) coefficient of the consent of the participants. For the time being, however, only few systems 
see participation as a bridge between formal lawfulness and the prevalence of the public interest 
and administrative efficiency and consider it so important that they include it in their APAs (such 
as in the USA; see Kovač and Sever 2014). Thus, for example, most European countries as well 
as the EU still have reservations as regards involving the public, or they allow only the partial 
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participation thereof (e.g., in the Aarhus Convention in relation to the environment) and only 
based on recommendations (for more, see Karpen and Xanthaki 2020). 

Participation is not an end in itself, but it creates better conditions for decision-making 
and — by involving the addressees of rules and acts — significantly increases the level of the 
acceptance and implementation of public policies. The purpose of the principle of participation 
in administrative relations is twofold: to protect the administratively defined rights and interests 
of stakeholders and to establish the true and objective factual situation relevant for a decision 
(evidence-based governance) (see OECD 2011). Public participation in administrative relations 
is even more important due to the party’s otherwise subordinate position to the — at least poten- 
tially — opposite public interest. Stakeholder participation in administrative relations is therefore 
a counterweight to the (otherwise) superiority of the public interest. After all, the importance 
of “procedural justice,’ for example in the USA and more so in Central Europe, shows the 
prevalence thereof over distributive justice or substantive outcomes (Tyler 2006: 5, 73ff). As 
long as people trust the procedural guarantees and the institutions that provide them, they are 
willing to subject themselves to the authorities, albeit to their own detriment, as fact-finding 
and evidence-taking procedures allow them to indirectly monitor and manage the outcome of 
the procedure. 

Administrative relations are necessarily conditioned by balanced respect for the public inter- 
est, based on the social values pursued by the legislature when adopting systemic and sector- 
specific laws: i.e., the level of institutional public governance. In implementing laws, public 
administration is bound by this framework (for more, see Raadschelders 2011; Coglianese and 
Kagan 2007; Peters and Pierre 2005). This means that when including the will of the citizens 
and social stakeholders (e.g., companies, non-governmental organizations), it must always assess 
which solutions are optimal for the (enacted) public interest. For the decisions to be legitimate, 
the administration must therefore consider both the results of the participatory processes and 
their placement in the legally defined public interest. However, research shows that procedures 
are flawed and the existing rules — e.g., governmental rules of procedure or resolutions — are 
misunderstood, not complied with, or excessively rigid (see Karpen and Xanthaki 2020). When 
the state, as a sovereign regulatory entity, makes binding decisions that are not complied with or 
are unilaterally conditioned, the effect on society is disruptive (Schuppert 2000: 777; Tyler 2006: 
163ff). These problems, therefore, need to be solved as insisting on obsolete administrative pro- 
cedural solutions leads to a deterioration of the situation, competitiveness, and a lower degree of 
democracy and efficiency of public governance. At the same time, there are good practices (for 
more on ADR, see Dragos and Neamtu 2014) that, after careful analysis of where and to what 
extent they are applicable to a particular regulation or type of procedure, could be adopted as a 
suitable regulatory framework. 

In order to achieve common principles and avoid any irregularities, the future codification of 
administrative procedure should be approached in two steps. First, it is necessary to determine 
the values and thus the minimum standards of all administrative procedures applicable to regula- 
tory, single-case, or any other acts. Then, the specifics of each type of procedure or relation to be 
determined thereby should be defined, with a clear distinction between general and individual 
or hybrid matters and between authoritarian (e.g., inspection) and service-oriented (e.g., educa- 
tion or health care) administrative activities. Such general codification or “administrative code” 
should contain: 


[C]ommon administrative principles, regardless of the type of administrative activity, 
including, in particular, participation, transparency, and administrative efficiency in the 


sense of achieving the set objectives of the sector-specific public policies; 
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principles and rules for different types of administrative processes, in particular for the 
adoption of administrative rules and single-case administrative decision-making, weighing 
between the inevitable formal legality and yet necessary responsiveness; 

basic rules on the judicial review of public administration, standardized for various 
administrative acts, in order to establish the rule of law and accountability; and 

evaluation mechanisms for assessing the consistency of the objectives of the regula- 
tions and their implementation in administrative areas, which strive for efficiency and 
accountability. 


Any APA should thus comprise more sound governance and less government and adminis- 
tration with a formalistic explanation of the rules governing administrative relations. A holistic 
approach with a broader and GG-oriented regulation in the APA can indeed go beyond that, 
both in individual countries and globally. 


Conclusion 


It can be concluded that the development of common principles and fundamental rights, 
together with a systemic approach to the general codification of all administrative processes, 
would contribute significantly to effective public governance and to democratic and efficient 
public administration. Given the lack of a holistic approach to fostering participation, for exam- 
ple, the currently applicable guidelines and regulations lack the capacity to ensure a consensus 
between the authorities and stakeholders and lead to a clear implementation gap. By including 
elements of participation, administrative procedure becomes more than a framework for pro- 
tecting the public interest and the rights of parties, as participation enables the wider flow of 
information and the institutionalization of advocating for socially relevant although not (yet) 
legally existing interests. In this context, the systemic codification of administrative procedures 
would also play a role in the efficient evaluation and development of administrative reforms and 
future public policies. 
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THE TRANSFORMATIVE 
POTENTIAL OF EVALUATION 
AS A POLICY TOOL 


Jill Anne Chouinard and Tamara Krawchenko 


As an applied domain of inquiry, evaluation is very much grounded in the current moment, 
shaped and influenced by a diversity of external circumstances and conditions in which con- 
flicting interests, perspectives and voices often compete for attention. In the 1970s, evaluation 
was considered one of the new and promising technologies of government designed to provide 
bureaucratic accountability and to assist policymakers in determining program success and fail- 
ure. While the field of evaluation has certainly become more methodologically diverse over 
the past 50 years, and collaborative and participatory approaches are now considered credible, 
its modernist legacy continues to influence the possibilities and parameters of practice. In this 
chapter, the authors situate the practice of evaluation within the current Canadian public policy 
environment and explore the reception and use of more democratic approaches to evaluation as 
a likely policy instrument today. 

“An open society becomes a closed society when only the officials know what is going on” 
(Cronbach and Associates, 1980, p. 95). 


Introduction 


Modern program evaluation emerged in the 1970s as a promising new “scientific” approach to 
research designed to provide bureaucratic accountability and to help American policymakers 
determine program success and failure (Fischer, 2003). While evaluation has never quite lived 
up to its modernist ideal, it has continued to play a technocratic role in public sector institutions, 
remaining a key player in current public management and accountability systems, becoming 
what Dahler-Larsen (2012) calls an “institution” in society. Our understanding of evaluation, 
particularly within public sector contexts, remains centred on evidence about “what works” 
(Richardson, 2013) and enmeshed with the bureaucratic ideals of good governance. Thus, while 
the field of evaluation has evolved over the past 50 years to now encompass diverse method- 
ological options, its modernist legacy continues to hold sway across much of the public sector. 

In a recent online discussion, Michael Quinn Patton portrayed evaluation as a “kaleidoscope,” 
stating 


You look at it one way and the pattern reveals a profession, another turn and it is a 
discipline, then turn again and a transdisciplinary image emerges, and yet another turn 
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and it takes on the pattern of an applied social science, then the pattern shifts with 
another turn and it becomes imagined as a technology, then an art, and still turning the 
kaleidoscope again, evaluation is science. In essence, evaluation is a many-splendored 
thing and in keeping with intersectional identity framing these days, evaluation has 
multiple identities and diverse brands. It’s all in how you look at particular patterns in 
particular light. It isn’t one thing, it is many things. 

(EvalTalk, October 9, 2019) 


These “multiple identities” means that evaluation is called upon to play a number of differ- 
ent roles, depending upon the program sponsor, the context, and whether its intended use is 
to render judgment, aid program improvement, generate knowledge, facilitate decision mak- 
ing, enable learning and capacity building, assist program development or provide account- 
ability or ongoing monitoring. Its kaleidoscopic nature also means that methodologically, 
evaluation can serve a technocratic function for accountability purposes or fulfil a learning 
function with aspirations tied to social democratic aims (Hanberger, 2006; Schwandt, 2002). 
These visions of evaluation, characterized by MacDonald (1974) as bureaucratic, autocratic 
and democratic, represent quite distinct paradigms, each defined by its own context of prac- 
tice, epistemological and theoretical foundation, rationale, utility, audience, evaluator role 
and stakeholder interest. 

Despite the methodological breadth of the field, the prevalent vision of evaluation still 
favours a technocratic approach based on the promise of impartiality, evidence, objectivity and 
accountability. Evaluation, by this definition, is valued for its perceived scientific and technical 
authority (House, 1993) and used as a way to legitimize government activities, ensure cost 
effectiveness and enhance managerial decision-making. Democratic approaches to evaluation, 
on the other hand, are designed to foster dialogue and promote active inclusion among a 
diversity of program stakeholders, with rationales linked to local ownership, empowerment, 
organizational and individual learning and program improvement (Cousins and Chouinard, 
2012). This characterization is purposefully stark. In the current public policy context, what 
many consider a context over reliant on a technocratic, empiricist approach to knowledge 
(Danziger, 1995; deLeon, 1994; Fischer, 2003; Stone, 2002), the reception and use of more 
democratic approaches to evaluation as a policy instrument remain an ongoing and persistent 
challenge (Fischer, 2003). While there is a significant body of research in the policy sci- 
ences dedicated to the inclusion of participatory and more democratic practices, the tension 
between the technocratic moorings of the field and its participatory potential remains distant 
(Fischer, 2000; Torgerson, 1986, 2003). 

In this chapter, we look critically at the challenges of introducing democratic approaches 
to evaluation as a policy instrument within the current public sector context. Our critique 
is focused on the instrumental role evaluation plays in supporting the current accountability 
function within public sector institutions. Specifically, we look at the apparatus of “good gov- 
ernance,’ as defined by our characterization of three “technologies” of evaluation and public 
policy related to technocratic approaches to knowledge construction and the use of indicators, 
targets, benchmarks, checklists and log frames, the overreliance on a positivist methodology, the 
increasing need for “experts.” We begin with a brief description of the current public policy 
environment, focusing on the dynamics and evolution of policy instrument design, selection 
and use. We then turn to a discussion of the three “technologies” that represent the evaluation- 
public policy nexus, with a focus on their epistemological, relational and political implications. 
We then shift to a discussion of collaborative approaches to evaluation, reflecting on its promise 
and potential in the current policy context. 
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Policy Paradigms: The Persistence of the Standard 
Model of Policymaking 


The diverse literature on policymaking agrees on one thing — policymaking today is complex. 
There are multiple and competing policy paradigms at play and, an ever-growing literature tries to 
make sense of how policies arise, persist and change and seeks to understand how ideas, institu- 
tions and interests shape them (Hogan and Howlett, 2015). In this multi-layered environment, 
the seemingly simple model of the policy-making cycle first conceived by Lasswell (1956) has 
had a lasting impact. It introduced the concept of the policy cycle with discrete stages flow- 
ing from agenda-setting, policy formulation, decision-making, implementation and finally to 
evaluation, in a feedback loop. Over six decades since its inception, the policy cycle approach 
has come under a weight of critique and, at the same time, has been elaborated, extended and 
reinvented, revealing its endurance as a useful way to think about policymaking. Its positivist 
orientations have been relaxed, and the discrete elements of the cycle have formed a heuristic 
to delve deeper into elements of the policy process, with many books organized along these 
elements — including this one. 

This model of the “policy sciences” has three main features: (1) it is consciously framed as 
problem-oriented; (2) it is necessarily multidisciplinary in order to address complex problems; 
and (3) it is deliberately normative or value oriented (rejecting behaviourism in the social sci- 
ences) (summarized from deLeon and Vogenbeck, 2007, p. 4). This last point requires elab- 
oration — while Lasswell was a stated positivist, he embraced the ultimately political nature 
of problem definition and agenda-setting while maintaining the importance of scientific and 
empirical objectivity in other elements of the policy cycle (Lasswell, 1970). One of the reasons 
for the endurance of the standard model is that it conforms to the dichotomy of politics/admin- 
istration post WWII whereby agenda-setting and policy formulation fall into the political arena 
and the remaining aspects to that of the bureaucracy (Jann and Wegrich, 2007). 

The field of public administration has long grappled with tensions between rationalist empir- 
icism and the value-laden nature of the policy process. The policy sciences have been critiqued 
for an overreliance on instrumental rationality and technocratic orientations (Deleon, 1994). 
Postmodernist critical policy studies have emphasised the need for policy analysts to embrace 
debate and to “subject moral and political issues to serious discussion, so that eventually, even 
if temporarily, the better arguments will prevail, providing us with our most reliable guides to 
action” (Danziger, 1995, p. 448). It has placed a lens on how power is constructed and upheld 
in the policy process, urging practitioners to “demystify” professional expertise and think about 
whose voices are privileged in the process (Schön, 2017). There has been a deliberative and 
participatory turn in public policy and administration, yet at the same time, rationalist models 
remain and are often reinforced by administrative protocols and accountability requirements. 
Reflexive and deliberative practices may be valued, but in day-to-day public administration, 
pragmatism often prevails. It can be simply too time consuming and complex to work in this 
way — easier to implement at the local scale than at a regional or national one (Bua and Escobar, 
2018). 

The literature on policy tools and policy mixes has brought nuance to the “black box” of 
policymaking, urging us to think about how choices about what to do and how to do it are 
made alongside understanding how the “thick” environment of existing policy tools interacts, 
complements, supplements and sometimes undermines policy goals and outcomes. What does 
it mean to think of evaluation as a “tool” within this process? Capano and Howlett (2020) 
have noted that the policy tools literature has focused on how tool choices are made from two 
perspectives: (1) an “input” lens that focuses on actors and interactions in the policy process 
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(instrument constituencies) and (2) an “output” lens that analyzes the content of adopted deci- 
sions and, specifically, whether the deployed instrument has achieved its stated goals. Evaluation 
has a role to play in both these approaches; however, as will be discussed, the methods that are 
used fundamentally structure the transformative potential of evaluation as a policy tool. 


The Evaluation-Public Policy Nexus 


The call for more democratic approaches to public policy evaluation dates back to the 1940s 
and the early days of the policy sciences field. Despite a few promising shifts towards a more 
pluralistic system over the years (deLeon, 1994), there remains what Torgerson (2017) refers to 
as a “conspicuous inattention to democracy” (p. 342) in current public policy circles. Instead, 
critics point towards an overreliance on a technocratic, empiricist approach to knowledge that, 
even today, seeks a value-free approach to public policymaking positioned outside the political 
arena (see Danziger, 1995; deLeon, 1994; Fischer, 2003). 

The challenges of democratizing the policy sciences are thus well documented (Richardson, 
2013), with some focused on identifying the conflict between technocracy and democracy (Gil- 
ley, 2017; Ryan, 2018) and others noting the methodological and epistemological challenges 
such a democratic shift might entail (Fischer, 2000). 

In this section, we describe three “technologies” of the bureaucratic state, all of which involve 
evaluation for legitimacy, accountability, methodological expertise and reporting. These three 
“technologies” are (1) technocratic approaches to knowledge construction (e.g., through the 
use of indicators, targets, benchmarks, checklists and log frames), (2) positivist methodologi- 
cal modes of inquiry and (3) methodological expertise. Although there is significant overlap 
and interconnection among these three mutually supportive “technologies,” our focus is on 
exploring the epistemological, relational and political implications of each as, we will argue, 
they ultimately limit the potential for more democratic approaches to evaluation and to public 
policy-making process more broadly. 


A Technocratic Approach to Knowledge and 
Knowledge Construction 


There is a robust and critical scholarship documenting the neo-liberal effects on governance and 
epistemology (see Espeland and Sauder, 2007; Merry, 2011; Shore and Wright, 2000, 2015). 
While evaluation does not always play a leading role, the “technologies” of evaluation are essen- 
tial to its success. Of note is the persistent reliance and faith in the value of numbers, as articulated 
and expressed in the form of indicators, targets, benchmarks and checklists: what Merry (2016) 
refers to as “indicator culture,” with quite distinct assumptions about what knowledge is and 
how it is constructed (Gildersleeve, 2017). While indicators give the impression of added sim- 
plicity, by reducing complex social phenomena to easily definable and measurable pieces, they 
ultimately create a distorted, depoliticized and decontextualized version of the world, essentially 
reconstructing an image or a concept of the world they are designed to measure (Dahler-Larsen, 
2012; Merry, 2016). Dahler-Larsen (2012, 2014) uses the term constitutive effects to describe how 
indicators function as a way of orienting action, discourse and attention to preselected evaluative 
indicators and criteria, and, in so doing, they create a particular socio-political version of reality 
that determines not only what can and should be counted but also how and by whom. 

As Dahler-Larsen (2014) explains, “[T]he indicator is embedded in a larger world view, a 
network of interpretations, in which its definition of activity makes sense” (p. 977). As Franklin 
(1990) argues, the tools have redefined the problem. Thus, while indicators, as codified metrics 
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of social and economic phenomena, purport to stand outside the fray of politics as a policy- 
making instrument and a vital “technology” of evaluation, their construction, orientation and 
use have critical epistemological, ontological, political and democratic consequences. 

As a formalized accounting system, these evaluation “technologies” are in widespread use 
for decision-making, monitoring, standard setting, improvement, reporting and controlling of 
almost all program activities and expenditures within public sector institutions. Whether locally 
or more globally, indicators have now become essential to evaluation and performance measure- 
ment as a model of public sector governance. This almost singular focus on performance mea- 
surement metrics narrows program and community understanding to predetermined measures 
of program success, confines program quality to measurable outcomes, reinforces ways of know- 
ing that already exist and that have been predefined and predetermined, circumscribes method 
selection to primarily quantitative methods and curtails stakeholder input and public dialogue. 
We live in a culture of compliance (Franklin, 1990) as, over time, we have become less willing to 
question the orthodoxy surrounding these metrics, less willing to question the implicit theories 
that underlie and frame their program logic. 

Despite its technological and rational facade, its apolitical cast, the current evaluation and 
performance management architecture (the “indicator culture”), it is not neutral, and its repre- 
sentation extends far beyond providing technical solutions. As Merry (2016) argues, “[R]ather 
than revealing truth, indicators create it. However, the result is not simply a fiction but a par- 
ticular way of dividing up and making known one reality among many possibilities” (p. 5). By 
paying closer attention to the way indicators are constructed and measured, evaluators become 
more aware of the reality about which they are making judgments, as well as the implications 
of those judgments. 


Positivist Methodological Approaches 


In evaluation, “what works” defines the gold standard and dictates methods used. In a recent 
global study of evaluator education from 1997 to 2017, LaVelle (2020) noted that the majority of 
programs (identified as advanced degrees, diplomas or certificates in evaluation) had a quantita- 
tive focus, with a number of courses in experimental/quasi-experimental methods or statistics. 
Courses in qualitative studies were rare. It is no surprise, then, that despite the plethora of fac- 
tors involved in designing an evaluation, the methodology of choice is most often quantitative, 
with experimental or quasi-experimental design considered the gold standard. This focus on 
quantitative measures comes at the expense of other important, relevant and perhaps more local 
criteria as the complexity of context is reduced to a narrow range of observable, measurable and 
recordable parameters, what Gordon et al. (1990) call a form of “methodological hegemony.” 
While quantification may give the illusion of scientific authority, lurking behind its seemingly 
apolitical cast is a profound transformation in how individuals, organizations and even coun- 
tries relate and how they are managed and governed (Shore and Wright, 2015). Numbers (and 
associated indicators of performance) are seductive (Merry, 2016) as they create a false sense of 
security and belief that everything relevant can be managed, controlled and measured; nothing 
is left to chance. 

The focus on quantitative measures and indicators of success also frames our understanding of 
public programs in terms of how they are defined, how they work, and how they can be made 
more effective and efficient, and, in so doing, they ultimately shape the parameters of public 
expectation, dialogue and discourse in ways that give the illusion of democracy (Schwandt, 
2009). Relationships in the field are thus structured around satisfying the plethora of account- 
ability demands rather than in working collectively and collaboratively with colleagues and 
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other stakeholders, a fundamentally antidemocratic admission when we consider the diversity 
of voices and perspectives that will remain unheard. The privileging of quantitative measures — 
what Stone (2002) frames as a “rationality project” — while purportedly unrelated to any political 
agenda reshapes our relationships, it prevents the inclusion of local experiences and perspectives, 
ultimately encouraging the further dominance of an “expert” elite (Fischer, 2009), further exac- 
erbating the divide between experts and citizens (Torgerson, 1986). 


Reliance on Expertise 


Concern with a growing class of professional experts is not new. As early as the 1920s, John 
Dewey worried that the need for expertise would lead to a further separation between the expert 
class and the public, making it increasingly difficult for citizens to participate in democratic life. 
Today, with ever-increasing technical sophistication and socio-political complexity in issues of 
public concern, we are witnessing a significant rise in the role of experts in the decision-making 
process, what Fischer (2009) calls the “age of expertise.” The field of evaluation is not immune 
to this discussion as the institutionalization of evaluation has led to calls for professionalization, 
credentialing, competencies and evaluator certification, all of which remain quite contentious, 
given the methodological breadth of the field. Of primary concern is that the current debate is 
centred primarily on the selection of competencies and an agreed-upon knowledge base, rather 
than on what professionalism means in evaluation and what evaluation’s role in society ought to 
be (Schwandt, 2018). Missing are questions about the purpose of evaluation, the role of evalu- 
ators, the values and principles of practice, the nature and limitations of the inquiry process etc. 
This lack of debate pushes the field further into the terrain of evaluator as technical expert, 
where the focus is on core knowledge, skills and competencies, thus narrowing and reducing 
the potential and practice of evaluation even further. 

Diminishing the role of the evaluator to “methodological technocrat” (Kuntz, 2015, p. 33) 
and the role of evaluation to serving a technocratic and legitimizing function within govern- 
ment, positions evaluation as fundamentally undemocratic. Specialized knowledge lies at the 
heart of professional practice. Access to this specialized knowledge (through training and educa- 
tion) will be restricted to only a privileged few, and, as a result, many other perspectives and 
voices will be excluded in evaluation. The fortunate few with technical skills will thus be further 
distanced from the majority of non-experts, further entrenching the division among people 
based on privilege, power, access and authority. This “expertocracy” (Fischer, 2009) restructures 
social relationships as it carries over into a hierarchy among “knowers” (Maguire, 1987). While 
“experts” represent only a small fraction of citizens, they become the “knowers” of the world, 
and all others the “nonknowers” (Hall and Tandon, 2017). Where one is positioned in the 
hierarchy of knowers is thus epistemologically, politically and democratically significant (Frank, 
2013) as these obstacles to inclusion can deny citizens the opportunity to participate simply 
based on who they, where they come from and whether they have the language skills or knowl- 
edge to engage in the policy-making process. The emphasis on epistemic privilege in the form 
of codified and specialized knowledge and in the promotion of “experts” constructs a hierarchy 
of knowers, ultimately creating what is potentially a further epistemic divide between evaluators 
and research participants — ultimately, a profoundly undemocratic act. 


Situating Collaborative Approach to Evaluation: A Brief Overview 


Participatory and collaborative approaches to research and evaluation have been around for 
decades, emerging in large part as a challenge to traditional and objectivist approaches to research 
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in the social sciences (Hall, 2005). In collaborative research, the goal is not to discover universal 
and generalizable principles, but rather to engage in active partnership with participants while 
working collaboratively to co-construct local narratives. The value of such an approach lies in its 
relational, dialogic and highly particularistic view of research and in the value and priority given 
to pluralist conceptions. It is precisely its highly engaged, relational stance that sets participatory 
and collaborative evaluations apart from other approaches, as well as the active involvement of 
multiple and often diverse stakeholders, in the engaged positioning of the evaluator and in the 
relationships that are created between evaluators and stakeholders. From a participatory per- 
spective, evaluators and stakeholders are inextricably linked in what Heron and Reason (1997) 
describes as “intersubjective space” that sees evaluators and stakeholders as active co-constructors 
in the creation of evaluative knowledge. 

Its roots can be traced to the early theoretical work of Paolo Freire, Antonio Gramsci and 
Jurgen Habermas, who were all in one way or another working to promote social justice and 
ameliorate historical injustice among the world’s most vulnerable populations. Although there 
is significant variation among collaborative approaches, they all go beyond traditional empiricist 
approaches to social inquiry and share epistemological roots in social constructivism and criti- 
cal theory (Heron and Reason, 1997), highlighting the importance of context, the value-laden 
nature of inquiry practice and the challenges of including multiple stakeholders in the co- 
construction of knowledge. 

A defining feature of participatory approaches to evaluation is the relationship or partnership 
between trained evaluators and program community members and stakeholders, all of whom 
work together in the production of evaluative knowledge. Depending on program and commu- 
nity contexts and evaluation purpose, approaches can vary with respect to the balance of control 
between evaluators and stakeholders in terms of decision-making, the level of diversity among 
stakeholders selected for participation and the level of stakeholder involvement in evaluation 
planning, instrument development, data collection, data analysis and reporting (see Cousins and 
Chouinard, 2012; Cousins and Whitmore, 1998). The key point here is that stakeholders play 
an active decision-making role throughout the process of evaluation, moving beyond the role of 
data provider to become active partners and collaborators in producing evaluative knowledge. 
By way of illustration, Arnstein’s (1969) ladder of participation depicts eight different types of 
participation across a broad spectrum, from nonparticipation and manipulation to informing and 
consulting, all the way up to partnership and citizen power and control. 

Over the years (see Cousins and Whitmore, 1998), consideration has been given to differ- 
entiating between evaluation and research, leading to questions about the goals of the inquiry, 
whose information needs and interests would be met and perhaps most importantly, whose 
interests are to be served (Cousins and Chouinard, 2012). These inquiries led to questions about 
the rationale or justification for adopting a collaborative approach, ultimately leading to a further 
distinction among three primary yet interconnected justifications. A political rationale is rooted 
in the ideals of social justice and based on a moral and normative sense of obligation involving 
the democratic inclusion, empowerment and emancipation of those who have traditionally been 
left out of the process of social and political change. 

Through participation in the evaluation, these stakeholders (often from systemically margin- 
alized populations) and other program community members learn to appreciate their circum- 
stances from a different perspective and to recognize oppressive forces at play. In Freirean terms, 
it is through such recognition that subsequent ameliorative action may be precipitated (Freire, 
1970). A philosophical motivation is based on principles of social constructivism, in which the 
centrality of context and the inclusion of multiple stakeholders in the production of knowl- 
edge are given priority. Through evaluators working hand in hand with program community 
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members, the co-construction of knowledge is grounded in historical, socio-political, cultural, 
economic and educational context, and as a result of this deeper epistemological interconnec- 
tion, deeper meaning of evaluative evidence and knowledge is achieved. 

A pragmatic justification is practical in orientation and based on the belief that increased par- 
ticipation will lead to the use of results to support program and organizational decision-making 
(Patton, 1997) and enhance organizational learning. The focus 1s on instrumental use of findings 
for discrete decisions, conceptual uses of evaluation findings for their educative function and, to 
a limited extent, the symbolic or persuasive use of findings to legitimize outcomes. As noted, 
these three rationales are not mutually exclusive, and emphasis on one or more justification will 
depend upon information needs, stakeholders, contextual and cultural exigencies and circum- 
stances. Despite the myriad rationales for selecting a participatory approach, its predominant 
use in the evaluation of programs is in culturally diverse communities, in Indigenous program 
and community contexts, and in evaluations conducted in international development settings 
(Chouinard and Cousins, 2009, 2015). In the next section, we turn to a discussion of the ten- 
sions between collaborative and technocratic approaches in public policy contexts. 


Discussion 


The foregoing discussion of the policy-making process highlights the complexity of social, polit- 
ical and institutional forces involved in knowledge and policy construction, as well as the role 
of diverse actors and systems implicated in producing knowledge for the public good. As Weiss 
(1999) has made clear, the creation, construction and adoption of policy are never straightfor- 
ward, particularly given the sheer number of different and often competing agendas and political 
interests and players involved. 

While evaluation has the potential to provide a diverse range of methodological possibili- 
ties, from serving an instrumental function to one that broadens public participation, within 
the public sector context evaluation is valued for its perceived scientific and technical author- 
ity (House, 1993) and used primarily as a management tool designed for accountability and 
decision-making purposes. Despite the multiple and diverse methodologies available, we see the 
widespread adoption and use of evaluation “technologies” as formal mechanisms for decision- 
making, monitoring, standard setting, improvement, reporting and controlling of program activ- 
ities and expenditures within public sector institutions. As an institutionalized and formalized 
activity, evaluation processes and practices provide an organizational legitimacy that is taken for 
granted, despite the significant epistemological, methodological and democratic consequences. 
As a way to highlight the challenges of adopting collaborative approaches in a policy-making 
context, in the following, we explore the differences (and tensions) between a collaborative 
approach and a technocratic approach to evaluation. 

As Table 33.1 illustrates, collaborative and technocratic approaches to evaluation are notably 
distinct, with contrasting historical roots, purposes, role of the evaluator, values and contexts of 
practice. With such divergent visions and practices, the questions then become “Who is evalu- 
ation in service to?” and “What is its purpose?” 

As a “tool” within the policy process, collaborative approaches have the potential to disrupt 
instrument constituencies, fundamentally reorienting the policy process to a broader range of 
policy instrument considerations. Unlike technocratic evaluation, collaborative approaches do 
not seek to preserve and maintain the status quo but to challenge established norms and practices 
and broaden the policy conversation to include the perspectives and voices of relevant (and often 
historically marginalized) communities who, in a democratic society, ought to be engaged in 
the conversation. 
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Consider the Canadian Truth and Reconciliation Commission’s (TRC) Call to Action in 
the context of the Canadian federal government’s reparations to Indigenous peoples, a commit- 
ment that foregrounds the need for collaboration in assessing the implementation and impact 
of the 94 Call to Action interventions. As Prime Minister Trudeau has stated, it is a time for 
renewed “nation-to-nation relationship, based on recognition of rights, respect, co-operation, 
and partnership” (Government of Canada, 2018). A relational approach to evaluation that values 
inclusion, voice, understanding and engagement would be salient in such program and policy 
contexts as the need to dismantle and disrupt past practice and build new relationships is essential 
for future planning and for measuring progress. In this context, social inquiry serves more than 
a technocratic accounting function. 

Further concerns about intergenerational equity, rising rates of poverty and the climate cri- 
sis also highlight the need for broader engagement and deliberation with relevant and affected 
constituencies in actively informing and shaping evaluation designs, processes and practices. The 
perspective of evaluation as a technology (Kushner, 2017) provides policymakers with a stan- 
dardized, prescriptive means of achieving putatively predictable outcomes. The legibility of the 
social world is read through numbers, statistics and indicators that represent the essence of social 
programs and processes. This is the vision of evaluator as technician. Technocratic approaches 
thus have profound epistemological and governance implications (Merry, 2016). 

Through engagement with a broader constituency, collaborative approaches can mobilize 
diverse perspectives, stories and histories and help provide critical questions and challenge cur- 
rent policy processes and outcomes. From a collaborative perspective, inclusion, engagement, 
voice and relationship represent how knowledge is constructed, by whom and for what pur- 
poses. This approach supports a vision of evaluation that is contextual, culturally and socially 
representative and fundamentally democratic. Technocratic approaches use the authority of sci- 
ence, objectivity and expert judgment to validate policy and remain in service to the bureau- 
cratic ideals of good governance. While this discussion is framed around the technical merits of 
experimental, quasi-experimental or quantitative methods versus other inquiry approaches, the 
fundamental issue has less to do with technical quality than with what Schwandt (2009) terms 
“epistemological politics” (the privileging of some knowledge and some knowers over others). 
While calls for more democratic approaches to policy are not new (deLeon, 1994; Fischer, 


Table 33.1 Differences Between Collaborative and Technocratic Approaches to Evaluation 


Collaborative 


Technocratic 


Historical roots 

Purpose 

Nature of Knowledge 
Values 

Stakeholder Involvement 
Role of Evaluator 


Method 
Use 


Context 


Political, response to modernist 
approaches 

Understanding, learning, 
empowerment, relationships 

Transformational, subjective, relational, 
constructivist, contextual/embedded 

Included, formative, local 

Direct, lived experience 

Engaged, responsive 

Mixed: qualitative and quantitative 

Conceptual, learning/educative, 
transformative 


Local, micro/macro influences 
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Rationalist, response to 
government services 

Explanation, prediction, 
confirmation, central control 

Technical, instrumental, realist, 
pre-determined, extractive 

Excluded, denied; gold standard 

Indirect, detached 

Independent, objective, observer 

Quantitative 

Instrumental (measure of 
effectiveness, efficiency, impact), 
symbolic use for accountability 

Universal standards 
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2003), failure to recognize the political dimension of policy analysis contributes to the use of 
evaluation as a technocratic instrument (Fischer, 2019). As a result, the democratic potential of 
evaluation as a policy-making instrument may never be realized. 


Conclusion 


In this chapter, we focused on the use of evaluation as a technocratic instrument in public sector 
contexts, focusing on the governance architecture supporting this vision of evaluation: the use 
of and reliance on predefined indicators, the use of quantitative methods as the gold standard 
for measures of performance and the need for technical expertise to translate complex metrics. 
The architecture supporting a technocratic approach to evaluation is embedded in the culture, 
politics and governance approaches in federal institutions. As Segerholm (2002) describes, 
“{E]valuation practice is part of larger political and managerial moods in today’s society and 
functions not only as a source of information to the public but as part of a larger web of different 
controlling, monitoring, and governing practices” (p. 98). 

From a technocratic perspective, evaluation is a key part of what it means to manage 
public policy and financial expenditures and a key part of accountability as it is currently 
understood. What is becoming ever more apparent is the explicit role evaluation now plays 
in the state’s system of governance (Ryan and Feller, 2009; Schwandt, 2009), not merely 
as passive bystander or reluctant companion but as an active partner with performance 
management in framing our expectations of government programs, policies and public ser- 
vices. Consequently, the possibility of widespread use of collaborative approaches to evalua- 
tion as a policy instrument seems remote. 

Nevertheless, examples such as the recent Call to Action in Canada around Indigenous 
rights cited earlier, which asks for regular monitoring and evaluation, highlights the poten- 
tial use of collaborative approaches to evaluation that are actively inclusive of Indigenous 
perspectives and voices and hold out hope for more effective and significant collaborative 
processes. 

As Bremner (2019) states, when it comes to evaluating the impact of the TRC Call to 
Action, we need to ask, “What stories need to be told and who will tell them? How will these 
stories speak to truth and reconciliation? How can we ensure that these stories will be heard?” In 
the current public policy context, there can be no stronger rationale for the use of collaborative 
approaches to evaluation. 
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POLICY TOOLS AND (NEW) 
GOVERNANCE ARRANGEMENTS 


Policy Mixes and Their Effectiveness 


Michael Howlett and Jeremy Rayner 


“New” Governance Arrangements (NGAs) are a kind of policy mix which has emerged as a 
lively topic in comparative policy studies and is often proposed as a solution to complex policy 
problems like environmental or health protection. However, assessing the merits and demerits of 
particular arrangements or instrument mixes is difficult. The chapter proposes a variety of tools 
to tackle the often-overlooked problem of identifying and inventorying the instruments found 
in instrument mixes and assessing their likelihood to produce optimal results. A framework is 
developed for evaluating the likelihood of successful implementation of NGAs that exploits the 
fact that new policy development is almost always constrained by previous policy choices which 
have become institutionalized. The degree to which this institutionalization has occurred is seen 
as variable and the implementation to depend on a number of well-understood processes such 
as increasing returns and other kinds of positive feedback, sunk costs, and incremental policy 
learning. The applicability of the framework is demonstrated in the context of NGAs found in 
the natural resource sector. 


Introduction 


An interesting quandary for comparative public policy analysis arises from the observation that 
the theoretical promise of a new policy instrument — such as, recently, emissions trading or envi- 
ronmental taxation — is rarely fully realized in practice (Vos 2007). While there are many reasons 
for this disappointing outcome, analysts have become increasingly aware that the real potential 
of a new policy instrument to improve policy outcomes lies not in its isolated application, but 
in the contribution it makes to an existing policy mix (Ganghof 2006; Chapman 2003; OECD 
2007; Keast et al. 2006). 

That is, most existing policy arrangements or regimes have developed incrementally in an ad 
hoc fashion over a relatively long period of time and contain a wide mix of policy instruments 
(Wilson 2000; Evers 2005; Gunningham and Sinclair 1999). These regimes sometimes contain 
a unifying overall logic but more often are the result of policy instruments and programmes 
being stacked on top of each other in a process which Thelen (2003) and Hacker (2005) have 
described as “policy drift” or “layering”. The results are arrangements of policy instruments 
that are both complex and costly to administer, often contain counterproductive instrument 
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mixes, and are difficult to change since key elements confer benefits on well-entrenched inter- 
ests (Beland 2007; Grabosky 1995; Pierson 1993). 

For these reasons, governments around the world grappling with new policy problems have 
become increasingly interested in crafting and adopting more carefully designed arrangements 
of instrument mixes or what are sometimes referred to as “New Governance Arrangements” 
(NGAs) (Howlett and Rayner 2006a, 2006b). New environmental policy initiatives dealing 
with issues ranging from pollution control to resource management, for example, are increas- 
ingly attempting to redesign entire policy regimes and avoid problems associated with layering 
and policy drift (Jordan et al. 2003, 2005). 

While NGAs mark a welcome departure from both incrementalism and entrenched pref- 
erences for one kind of instrument to the exclusion of all the others (Howlett 2005), NGAs 
are also not without problems of their own. Some of these problems are political and relate to 
implementation difficulties, such as weaning key actors off subsidies or re-regulating critical 
sets of social and economic activities against opposition from those actors benefiting from the 
status quo. Others, however, are analytical, most notably those connected with the identifica- 
tion of optimal policy designs and the avoidance of sub-optimal outcomes. Policy analysts have 
often been remarkably cavalier in their descriptions of the nature and limits of particular policy 
regimes and in their identification of the different policy instruments in a regime. In what fol- 
lows, we discuss both these issues in the design of integrated policy strategies (Meijers and Stead 
2004; Stead et al. 2004). 


Policy Instrument Mixes 


Policy instruments are techniques of governance which, one way or another, involve the utiliza- 
tion of state resources, or their conscious limitation, in order to achieve policy goals. They are 
the “tools of government”, the mechanisms and techniques used to implement or give effect to 
public policies (Salamon 2002). Analysis of the connection between policy regime effectiveness 
and instrument choice has a reasonably long history, but the focus of attention has changed at 
least three times. 

“First generation” scholars studying the tools of government were concerned largely with the 
analysis of business-government relations and with the effects of state regulation and economic 
policy formation on business efficiency. Although internecine debates between neo-classical and 
welfare economists over the concept were sharp, first generation instrument choice economists 
concentrated their efforts on identifying the market failures which would justify government 
intervention in market exchange and the possible governance techniques which could “correct” 
those failures (Bator 1958; Zerbe and McCurdy 1999; Breyer 1979; Zeckhauser and Schaefer 
1968). Other scholars, however, credited political rationales — such as ideological propensities, 
partisan electoral calculations, credit-claiming and blame-avoidance behaviour, and others — as 
the main explanation of instrument selection and regime effectiveness (Lowi 1966; Wilson 
1974; Trebilcock and Hartle 1982; Salamon 1981; Weaver 1986; Majone 1989). Public policy- 
makers were not generally thought to be driven by questions of theoretical purity but rather by 
a more overt political calculus. 

The first generation of early instrument analyses suffered from three main problems. First, 
studies tended to promote a misleading view of either the purely technical or purely political 
nature of instrument choices. Second, they tended to portray instrument choices in stark, “good 
and evil” terms, embracing, for example, “good” pro-market choices and “evil” non-market 
ones (Woodside 1986). And third, they contributed to a growing gap between complex admin- 
istrative practices on the ground and overly simplistic theoretical discussions and inquiries. 
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Not all early studies shared these characteristics, of course, and some analysts presented more 
complex and nuanced models and analyses of instruments and instrument choices (see, for 
example, Bressers and Klok 1988; Hood 1986). Building on the base of case studies and insights 
developed in these works, second generation students of instrument choice attempted to develop 
more policy-relevant models of instrument selection processes examining, for example, the role 
played by policy networks in instrument deliberations and choices (van Nispen and Ringeling 
1998; de Bruijn and Hufen 1998; Bressers and O”Toole 1998). Although it was acknowledged 
that, in some circumstances, governments might well choose particular instruments based on 
their technical efficiency and theoretical appropriateness, it was argued that this was likely to 
occur only in very specific circumstances — such as take place occasionally in areas such as fiscal 
and monetary policymaking where technical experts can prevail in policy deliberations (Markoff 
and Montecinos 1993). 

These second generation studies were a great improvement on first generation work but 
also featured some significant limitations of their own. First, they tended to focus on either 
those instruments designed to affect goods and service production and delivery in society — 
“substantive instruments” — or those instruments designed to alter policy processes — “procedural” 
instruments — and ignored their interrelationships (Howlett 2000, 2004). And secondly, and 
relatedly, in almost all cases, they also focused exclusively on single, discrete instances of instru- 
ment selection, even while acknowledging that instrument choices were often made in “bun- 
dles” or accumulated into such bundles or “mixes” over time (Hood 2007). 

This led to a “third generation” of work on instruments which attempted to overcome 
these limitations and apply the models developed by first and second generation theorists to 
the question of policy instrument mixes and especially to the potential to develop optimal 
policy instrument designs in complex multi-instrument settings (Grabosky 1994; Gunningham 
and Young 1997). This latter work represents an effort to correct many of the flaws of first and 
second generation thinking and addresses the disjuncture between administrative practice and 
instrument analysis which was a recurring feature of earlier work on the subject (Eliadis et al. 
2005). 

Third generation work on instrument choice is especially germane to the analysis of many 
so-called “New Governance Arrangements”, since NGAs represent efforts at integrated policy 
design and implementation revolving around the construction of policy mixes expected to opti- 
mize governments goals. NGAs, that is, are policy mixes designed as integrated strategies and 
are specifically intended to address the perceived shortcomings of previous, more ad hoc policy 
regimes. 


Design Principles for Policy Mixes 


What, then, are the principles upon which policy mixes — and thus NGAs — can be designed 
to ensure integration and optimality? Some basic principles were proposed by proponents of 
“smart regulation” in the late 1990s, such as Neil Gunningham and his colleagues. They sug- 
gested policymakers should: 


1. Consider the full range of policy instruments available, 

2. Employ a mix of policy instruments carefully chosen to create positive interactions with 
each other and to respond to particular, context-dependent features of the policy sector, 

3. In the context of continuing pressures on governments to do more with less, consider 
incentive-based instruments, various forms of self-regulation by industry, and policies that 
can employ commercial and non-commercial third parties to achieve compliance, and 
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4. Not overlook procedural policy instruments such as information instruments and the vari- 
ous techniques of network management. 
(Gunningham et al. 1998) 


While helpful, these suggestions are heavily influenced by Gunningham’s specific interest in 
environmental regulation and are more like maxims or “rules of thumb” than generalizable 
principles which could allow policy planners and managers to design optimal arrangements in 
a variety of different contexts. Appropriate policy design requires (1) specifying what kinds of 
tools are available to be mixed and (2) understanding the factors which allow us to designate any, 
and any particular, mix as integrated. These two questions are addressed next. 


What Tools Are Available? Elements of a Policy Mix 


As the chapters in the introductory section of this book have set out, first generation efforts to 
systematically study policy instruments produced several useful taxonomies of policy instru- 
ments (Tupper and Doern 1981; Vedung 1997) whose employment helped shed light on the 
reasons behind discrete instrument choices (e.g., Linder and Peters 1989, 1990, 1992). The tax- 
onomies were especially effective in identifying shifts in patterns of instrument choices, such as 
those associated with the waves of privatization and deregulation which characterized the 1980s 
and 1990s (Howlett and Ramesh 2003). 

Taxonomies, such as the one first developed by Christopher Hood (1986; see also Ander- 
son 1977) utilizing a “‘statecraft” rubric can also be used to identify the basic elements of any 
policy mix. Such taxonomies group instruments together according to which specific govern- 
ing resource they rely on for their effectiveness — in Hood’s case, “nodality” (or information), 
authority, treasure, or the organizational resources of government. Hood’s schema provides an 
overall template for assessing the potential components of any policy instrument mix (Table 34.1). 

Although policymakers are seemingly confronted with a number of choices of possible 
instruments in creating their strategies, the contribution of second generation instrument studies 


Table 34.1 A Taxonomy of Eight Basic Policy Instrument Components of a Policy Mix 


Principal Governing Resource Used 


Nodality Authority Tieasure Organization 
Substantive Advice Regulation Grants Administration 
General Purpose of Training Self-regulation User charges Public enterprises 
Instrument Use Reporting Licences Loans Policing 
Registration Census taking Tax credits Consultants 
Polling Record keeping 
Procedural Information Treaties Interest group Conferences 
provision/ Advisory funding/ Commissions of inquiry 
withdrawal comumittees/ creation Government 
commissions reorganizations 


(Cells provide examples of instruments in each category.) 


Source: Adapted from Christopher Hood, The Tools of Government (Chatham: Chatham House, 1986): 
124-125 and Michael Howlett, “Managing the ‘Hollow State’: Procedural Policy Instruments and Modern 
Governance,” Canadian Public Administration. 43, no. 4 (2000): 412-431 
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was to reveal that governments often repeatedly choose from a much more limited range of 
instruments from all the options. They set out a model of the eight basic types of instruments 
from which any policy mix is constructed. Each type is, of course, a shorthand expression for a 
variety of different tools and strategies, and the set may seem an unacceptable simplification of 
a much more complex reality. 

The first and second generation work also noted a distinct tendency for governments to 
develop an “implementation style” in various sectors and to stick with this style for quite some 
time (Kagan and Axelrad 1997; Howlett 2002, 2005). These implementation styles are composed 
of a combination, or mix, of substantive and procedural instruments, at minimum two. Hence, 
for example, the well-known implementation style found in many US policy sectors, dubbed 
“adversarial legalism” by Robert Kagan (1997, 1991), is composed of a preferred substantive 
instrument (regulation) and a characteristic procedural one (Judicial review) based on widespread, 
easily accessible legal procedures. Similarly regulations (a substantive tool) are often accompanied 
by regulatory advisory boards and/or various kinds of oversight processes (both procedural tools). 

The common presence of implementation styles highlights two important aspects of policy 
mixes that are relevant to contemporary policy instrument studies: (1) that they usually involve both 
substantive and procedural elements and (2) that the exact pairing of instruments has an important 
historical dimension linked to the kinds of governance arrangements found in a jurisdiction. 

This implies, among other things, that any assessment of the adequacy, coherence, or optimal- 
ity of instrument choices within an instrument mix requires that the specific features of particular 
mixes be identified but also how these elements have evolved over time (Howlett et al. 2006). 


What Factors Influence Optimality? Coherence and Consistency 
as Goals of Integrated Strategies 


During the 1990s, governments grappled with unfamiliar problems in a context in which they 
often found themselves required or called upon to do more with less. Policymakers became 
increasingly aware of the importance not just of expanding the range of instruments available to 
them but also of experimenting with new instrument mixes in many sectors, as well as with efforts 
to rationalize or consolidate older ones. Such “New Governance Arrangements” have been devel- 
oped in contexts as diverse as health assessments, national forest programmes, integrated coastal 
zone management (ICZM), safety cultures, integrated water management, and others (Hippes 
1988; Knoepfel 2007; Miller 1990; Bode 2006; Briassoulis 2004; Stead et al. 2004). 

The result has been a new kind of institutional design, the attempt to develop “integrated 
strategies” (IS) whereby, in addition to the substantive policy objectives that they pursue, gov- 
ernments also attempt to create or reconstruct a policy domain with coherent policy goals and 
a consistent set of policy instruments that support each other in the achievement of the goals. 

NGAs are intended to combine policy instruments and their settings in new ways so that 
multiple instruments support rather than undermining one another in the pursuit of policy 
goals. NGAs also attempt to integrate existing, and sometimes competing, policy initiatives into 
a cohesive strategy; to coordinate the activities of multiple agencies and actors; and, generally, to 
substitute a holistic approach to a problem for one that has decomposed policy into a set of mul- 
tiple and apparently unrelated problems and solutions (Briassoulis 2004, 2005; Stead et al. 2004). 

As discussed earlier, the analytical and practical challenges to understanding and improving 
NGA designs are twofold. First, the elements of old mixes must be identified and supplemented 
or replaced with new elements which are more coherent and consistent. Second, the recon- 
structive effort behind the creation of new designs demands a sophisticated analysis of policy 
dynamics and instrument choice which must deal not only with the technical questions of 
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replacing an existing set of policy instruments with a new, more integrated one but also with the 
political challenges of so doing. 


The Dynamics of Instrument Choice and (New) Governance Arrangements 


The main practical challenge facing integrated NGA designs is that they almost never begin 
with a clean slate. In fact, most NGA designs are conceived as a result of dissatisfaction with the 
incoherent goals and uncoordinated instruments that characterize an existing set of policies over 
two or more domains (May et al. 2005). The potential outcomes of NGA can be represented in 
the following way (see Table 34.2): 


Table 34.2 Typology of New Governance Arrangements According to the Relationships Between Goals 
and Means They Embody 


Instrument Mixes Are 


Consistent Inconsistent 
Multiple goals are coherent optimal ineffective 
incoherent misdirected failed 


The variety of possible outcomes is explained by the fact that new policy development is 
usually constrained by previous policy choices which have become institutionalized. The degree 
to which institutionalization has taken place — and hence the severity of the constraint it places 
on policy change — is variable and depends on a number of well-understood processes such 
as increasing returns and other kinds of positive feedback, sunk costs, and incremental policy 
learning. Following Hacker, we propose that efforts to create an optimal IS design that pursues 
multiple but internally coherent goals using multiple but consistent policy instruments can fail 
in three main ways: layering, drift, and conversion (Table 34.3). 

Layering is the one possible way to try to create an NGA but difficult to accomplish as it 
means adding new goals and instruments without abandoning previous ones, often leading to 
both incoherence amongst the goals and inconsistency with respect to the instruments. Many 
NGAs suffered this fate; for example, efforts at integrated coastal zone management (ICZM) 
which failed when powerful interests accepted new policy arrangements only if they could keep 
favourable goals, instruments, and settings in the new design, such as unsustainable fishing quo- 
tas to support an industry (Howlett and Rayner 2006a, 2004). 

Drift allows the goals of the policy to change without changing the instruments, which again 
is difficult to control or manage since the old tools may become inconsistent with the new goals 
and most likely ineffective in achieving them. Examples of this process are many, with many 


Table 34.3 Typology of NGAs According to Their Relationship With Existing Policies 


Instrument Mixes Are 


Consistent Inconsistent 
Multiple goals are coherent integration drift 
incoherent conversion layering 


446 


Policy Tools and (New) Governance 


analysts observing such processes at work in, for example, welfare state transitions from “welfare 
to workfare” in which policy goals are coherent (end welfare as we know it), but the tools used 
to address them are inconsistent or counterproductive (Evers 2005). 

Conversion, on the other hand, is the attempt to change the instrument mix in a more 
tractable policy domain in order to meet new goals in a domain where change is blocked by 
“converting” an existing tool to a new use. Building integrated land management (ILM) out of 
a protected areas strategy, for example, or attempting to use school vouchers to address aspects 
of social welfare policy, exhibit some of the problems associated with conversion (Miller 1990). 

Designing optimal integrated strategies, then, involves ensuring that sub-optimal outcomes 
from processes such as layering, drift, and conversion are avoided. All these outcomes and pro- 
cesses are apparent in a notable recent effort to develop a coherent NGA in the forest sector: the 
development in Europe and elsewhere of “national forest programs”, or NFPs. The case study of 
NFP development that follows provides an example of how to address the general challenges of 
assessing NGAs developed in circumstances in which better policy integration is a key objective. 


An NGA Case Study: The Development of National Forest Programs 


The idea of providing overall coordination and monitoring of the disparate elements of for- 
est policy through a new governance arrangement has been part of ongoing efforts to develop 
national forest programs in many countries. This effort first took shape during the ultimately 
unsuccessful attempt to negotiate a legally binding international convention on sustainable for- 
est management in the 1990s. Although those negotiations failed to produce a convention, the 
UN-sponsored IPF/IFF process did result in an “action plan” towards more sustainable forest 
practices which, though nonbinding, proved influential. The action plan contained, among 
other items, a section on attaining “progress through national forest and land-use programmes” 
in which countries were exhorted to “[d]evelop and implement a holistic national forest pro- 
gramme which integrates the conservation and sustainable use of forest resources and benefits in 
a way that is consistent with national, sub-national and local policies and strategies” (Humphreys 
2004). 

There was a patchwork of responses from different governments to the NFP idea, depend- 
ing on how invested they were in the existing governance arrangement they had in place and 
whether or not these could be changed through layering, draft, conversion, or replacement. In 
some cases, countries were early adopters of formal documents which they designated as an NFP 
but which, upon examination, largely failed to deliver on the goals of the action plan. 

In other cases, countries deliberately did not adopt a formal NFP, arguing that their existing 
forest policies were already sufficiently well integrated. The issue rapidly became politicized as 
INGOs derided the former group as merely paying lip service to the NFP idea while criticizing 
the latter as policy laggards. 


Governance Arrangements and NFPs 


The two key constitutive elements of an NFP noted by European researchers which allow NFPs 
to be examined as examples of efforts to put into place new governance strategies different 
from the traditional regime of regulation and subsidy for timber production purposes are (1) 
mechanisms enabling participatory deliberation and conflict resolution and (2) mechanisms for 
intersectoral coordination and policy learning. That is, in addition to traditional industry supply 
and production regulation and market promotion activities, NFPs also contain procedural policy 
tools which allow or encourage forms of autopoietic network management. 
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Table 34.4 Four Main Types of NFPs 


INPUTS/PROCESS 
NFP Mechanisms for Participation and Conflict Resolution 


NFP Mechanisms for Formal Informal 
Coordination and Policy Learning 


OUTPUTS Substantive Classical or “designed” NFP = Output-oriented NFP 
Symbolic Legitimizing or “process- Rhetorical NFP 
oriented” NFP 


Source: Modeled after Americo Carvalho Mendes’s “Implementation Analysis of National Forest 
Programmes” in Peter Glueck and Johannes Voitleithner (eds.) NFP Research: Its Retrospect and Prospect, 
Publication Series of the BOKU Institute of Forest Sector Policy and Economics, vol 52, Vienna 2004, 
pp. 31-46 


The former involve a variety of procedural instruments such as the creation of advisory com- 
mittees, mediation and arbitration provisions, and interest group facilitation which can be either 
formal — that is, set out and established in legal and regulatory frameworks — or more informal 
in nature. These tools are different from many existing governance arrangements which called 
for closed-door bargaining or negotiation between governments and industry in determining 
the content and character of a forest policy regime. 

The coordination and policy learning mechanisms also upset existing arrangements, involv- 
ing various forms of cross-sectoral environmental or industrial planning activities, taking into 
account additional resource and environmental considerations and actors than traditional industry- 
driven governance arrangements. 

Both these efforts can remain largely symbolic — that is, relegated to overall policy statements 
and general design principles — or can be substantive in nature — that is, involving the establish- 
ment of multi-sectoral committees and implementation agencies (Howlett and Rayner 2006a, 
2006b). 

Thus, as Table 34.4 shows, in addition to the “classical” NFP — the kind originally envisioned 
in the international negotiations, where formal consultative processes alter existing governance 
arrangements and produce substantive output change — there are a variety of other possibilities. 
One possibility is the “output-oriented NFP”, in which a more informal and less holistic plan- 
ning process is combined with the same substantive outputs envisioned in the original NFP idea. 
Other possibilities include the “legitimizing” or “process-oriented NFP”, which creates new 
formal processes but only symbolic outputs, and the purely “rhetorical NFP”, which contains 
neither substantive outputs nor formal processes and is an NFP in name only. In both these latter 
cases, traditional instrument mixes will continue largely in effect. 

That is, classical NFPs are expected to be governance arrangements which result in a consis- 
tent set of instruments and coherent goals, while failed NFPs are those without either. Output- 
oriented NFPs, on the other hand, risk carrying over instruments from a previous policy regime, 
such as bilateral government-industry standard setting, which can prove inconsistent with new 
instruments aimed at improving stakeholder participation or transparency for consumers. NFPs 
with incoherent goals, such as sustainable forest management and maximizing timber produc- 
tion, are likely to be legitimizing at best, even if they feature relatively consistent instrument 
mixes. 

As Glueck and Voitleithner (2004) and Howlett and Rayner (2006a) have shown, these types 
also correspond very closely to the development processes set out in Table 34.5. 
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Table 34.5 Typology of NFP New Governance Arrangements According to Their Relationship With 
Existing Policies 


Instrument Mixes Are 


Consistent Inconsistent 
Multiple goals are coherent Classical NFP Output-oriented NFP 
incoherent Legitimizing NFP Rhetorical NFP 
Conclusion 


Theories of policy instrument choice have gone through multiple “generations” (Goggin et al. 
1990; O’ Toole 2000) as theorists have moved from the analysis of individual instruments (Sal- 
amon 1981, 2002) to comparative studies of instrument selection (Howlett 1991; Bemelmans- 
Videc 1998; Peters and Van Nispen 1998; Varone 2000) and the development of theories of 
instrument choice (Trebilcock et al. 1982; Hood 1986; Linder and Peters 1989; Howlett 2004). 
Theorists, administrators, and politicians have expanded the menu of government choice to 
include both substantive and procedural instruments and a wider range of options of each and to 
understand the important context-based nature of instrument choices (Howlett 2000). “Third 
Generation” instrument choice theory moved beyond tool selection per se to address a series of 
concerns involved in designing and adopting optimal “mixes” of instruments in complex deci- 
sion-making and implementation contexts or governance arrangements (Bressers and O’Toole 
2005; Eliadis et al. 2005). 

The “new” governance arrangements which resulted — such as NFPs — are contemporary 
examples of complex policy designs which require analysis in order to understand the modali- 
ties of their development and likelihood of success. The key to understanding the relationship 
between such governance arrangements and policy tools, we have argued, is a more sophis- 
ticated theory of instrument choice, one that takes into account both the manner in which 
current decision-making occurs within the context of rounds of previous decisions and the fea- 
sibility of the ultimate ambition to provide a mix of mutually supporting policy instruments. As 
our NFP example demonstrates, conversion, layering, and drift are all policy change processes 
which interact with existing and proposed governance arrangements and can cause major chal- 
lenges to the achievement of integrated policymaking. 
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MEASURING POLICY 
INSTRUMENT INTERACTIONS 
IN POLICY MIXES 


Surveying the Conceptual and 
Methodological Landscape 


Moshe Maor and Michael Howlett 


Resolving a complex policy problem often requires a mix of policy instruments and thus the 
identification of the most promising instrument combination. However, the relevant terminol- 
ogy of instrument interactions in a policy mix has not been standardized, hindering a straight- 
forward identification of superior instrument combinations. To address this challenge, the 
chapter defines the terminology necessary for detecting three different possible policy instru- 
ment interactions — namely, synergistic, counterproductive, and additive effects. It identifies two 
approaches to analyzing instrument mix effects: the “effect-based” and the “effort-based” meth- 
ods. It then discusses the practical advantages and limitations of each approach and elaborates on 
key methodological issues that policy scholars and practitioners face at each step of developing 
a new policy mix. 


Introduction 


Individual policy instruments cannot solve complex policy problems. Most are beset by intrin- 
sic limitations that restrict their maximum contribution to the realization of policy goals. This 
problem led policy scholars to study bundles or portfolios of policy instruments (e.g., Howlett 
and Rayner 2007; Capano and Howlett 2020; Feindt and Flynn 2009; Kay 2007; Larsen et al. 
2006), and some have recently addressed the empirical analysis and conceptualization of policy 
mixes as systems of interconnected elements susceptible to simultaneous mutual or reciprocal 
interactions (del Rio 2010; Lecuyer and Bibas 2012; Rosenow et al. 2016; Trencher and van der 
Heijden 2019; Attwell and Navin 2019). This chapter tries to delve deeper into what it means 
for policy instruments to interact. 

Instrument interconnection or interactions occur in policy mixes when two or more policy 
tools are included in a combination that alters the way one or more of these instruments con- 
tribute to the realization of policy targets. These interactions can be multi-level (e.g., Howlett 
and del Rio 2015; Goyal and Howlett 2021) and cross-sectoral (e.g., Mantino and Vanni 2019; 
Boonekamp 2006) and may result in expected and unexpected effects (e.g., feedback effects, side 
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effects) across a wide range of contexts, making their analysis and implementation challenging. 
Evaluating policy mix differences requires a clear understanding of the nature of each tool as well 
as the nature of its relationship(s) to others. Indeed, this constitutes a critical element of policy 
design (Howlett and Mukherjee 2018). 

In order to ensure a design’s success, the definitional and methodological grounds underlying 
the evidence gathered on these interactions must be solid so that a conclusion that one instru- 
ment mix is significantly superior to another is well founded. Unfortunately, current research 
features a variety of competing means and methods, as well as concepts and vocabulary devel- 
oped to address these issues, and this confusion is hindering the clear exposition and understand- 
ing of the central problematic and its resolution. The overlaps and lacuna in the policy design 
field today are more pronounced than ever before because instrument combinations form the 
mainstay of current policy solutions for numerous complex policy problems, and synergistic 
instrument mixes are preferred in policy mixes for many policy problems. It is time to deepen 
our knowledge of synergistic, counterproductive, and other kinds of instrument interactions 
which are a feature of numerous policy mixes (e.g., del Rio 2010; Fernandez-i-Marin et al. 
2021; Grabosky 1995; Justen et al. 2013; Leplay and Thoyer 2011; Trencher and van der Heij- 
den 2019). 

Towards this end, this chapter provides the minimal set of concepts necessary to understand 
and measure instrument-instrument interactions, distinguishing between three different kinds 
of instrument interactions and dividing the methodologies used to assess them into effect-based 
approaches and effort-based approaches. It thereafter discusses the respective advantages and 
limitations of these two approaches and elaborates on key methodological issues and challenges 
that policy scholars must face and overcome at each step in the analysis and design of instrument 
mixes. 


Existing Concepts of Policy Tool Effects and Interactions 


A few concepts have entered the policy lexicon to help discern the key effects of multiple 
instruments in a policy mix. The terms synergy and counterproductive effects are often used to 
justify choices concerning the number of instruments required for the efficient attainment of 
a policy goal or goals (Rogge et al. 2017; Trencher and van der Heijden 2019). Both figured 
prominently, for example, in a recent discussion around how to avoid potentially under- or over- 
designing a policy mix (Maor 2020). 

Del Rio (2010), Rosenow et al. (2016), and Lecuyer and Bibas (2012), for example, have 
examined whether tool combinations are complementary, neutral, conflicting, or overlapping in 
the areas of environment and energy. Trencher and van der Heijden (2019) have used an adap- 
tive theory approach to identify complementarity advancing measures. And Attwell and Navin 
(2019) have offered a framework that emphasizes differences in mixes related to scope (which 
vaccines to require), sanctions (which penalty to impose), severity (how much of the penalty to 
impose), and selectivity (how to enforce or exempt people from vaccine mandates). Synthesized 
information about how these elements interact at the level of individual action could be thereaf- 
ter combined to a distinct attribute — saliency — which “identifies the magnitude of the burdens 
the state imposes on those who are not vaccinated” (Attwell and Navin 2019: 979). 

“Synergies” reflect the situation in which individual instruments combine in unexpected, 
nonlinear ways to enhance overall instruments mix effectiveness while “counterproductive” 
effects are those in which instruments combine to detract from overall instruments mix effec- 
tiveness. Among these interactions, synergy is a highly pursued goal of instruments mix devel- 
opment. Counterproductive effects in policy settings are less so (Trencher and van der Heijden 
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2019). Although, as we will elaborate on later, these effects can, at times, be desirable with 
regard to policy effectiveness. 

These terms describe critical elements of policy mixes — especially concerning “how well the 
elements of the policy mix are aligned with each [other], thereby contributing to the achieve- 
ment of policy objectives” (Rogge and Reichardt 2016: 1626). Individual instruments in a 
policy mix can be considered consistent when they work together to support a policy strategy: 
“They are inconsistent when they work against each other and are counter-productive” (Kern 
and Howlett 2009: 396). Note, however, that inconsistency does not necessarily lead to coun- 
terproductive effect; it can also have neutral effects: that is, having no strongly marked or positive 
effects. 

In total, there are three types of interactive effects among policy instruments that are relevant. 
These include additive effects (Boonekamp 2006; Justen et al. 2013; Justen et al. 2013; Yi and 
Feiock 2012), as well as the counterproductive effects and synergies mentioned earlier (Lecuyer 
and Bibas 2012; Philibert 2011; Trencher and van der Heijden 2019). 

There is indeed a broad consensus in the policy design literature that not all the effects of 
instruments in a policy mix are inherently complementary (e.g., Boonekamp 2006; del Rio 
et al. 2011; Grabosky 1995; Gunningham and Grabosky 1998; Gunningham and Sinclair 1999; 
Howlett 2017; Tinbergen 1952), that some policy designs generate counterproductive responses 
from policy targets (e.g., Schneider and Ingram 1988, 1990a, 1990b, 1993, 1997, 2005), that 
some policy portfolios may be superior to others because they offer a reinforcing or supplement- 
ing effect (e.g., Hou and Brewer 2010), and that some tool combinations may be unnecessarily 
duplicative in one context but advantageous in another (e.g., Braathen 2005, 2007; Mantino 
and Vanni 2019). 

Counterproductive effects are manifested, for example, when command-and-control regu- 
lation is used alongside voluntary compliance (Grabosky 1995), with each undermining the 
effectiveness of the other. This differs from more complementary additive effects which occur, 
for example, when command-and-control regulation to minimize undesirable modes of behav- 
ior is employed alongside financial incentives to promote more desirable ones by layering both 
incentives and disincentives together and having them both pull in the same direction (Hou and 
Brewer 2010). 

An important element in current policy design thinking, therefore, is to try to maximize 
additive efforts that supplement each other while minimizing counterproductive ones, and per- 
haps neutralizing situations wherein instrument combinations are “developed without any sense 
of an overall conscious design” (Howlett and del Rio 2015: 1235) and fall into disrepair or 
unintended contradictions (Daugbjerg 2009; Hou and Brewer 2010). 


Existing Measures for Assessing the Character of 
Policy Mixes: Density and Intensity 


Central to this analysis is the need for a clear(er) understanding of the precise kinds of interac- 
tions between the policy instruments arrayed in specific policy mixes: that is, how instruments 
relate to each other in specific contexts when they are combined in specific ways, and what 
impact this combination has on their contribution to the realization of policy goals. But what 
makes existing measures of policy mixes inadequate? 

Two measures in particular feature in the current literature: density, or the number of pol- 
icy instruments found in a mix, and intensity, or the manner in which those instruments are 
deployed in either a strong or a weak fashion. While density is a straightforward concept, inten- 
sity is more problematic as a wide range of intensity measures are possible, such as how many 
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objectives a mix is expected to achieve, the manner in which the instruments needed to reach 
these objectives are resourced, or the benefits and burdens affecting the target populations they 
entail (Bobrow 2006; Eliadis et al. 2005; Schneider and Sidney 2009). 

Both measures were developed taking into account Howlett and Cashore’s (2009) argu- 
ment that all policies are composed of several distinct elements found at three levels of 
abstraction: the overall abstract goals of policies and governance preferences of enacting 
governments, the kinds of objectives and instruments used to achieve these goals, and the 
settings and calibrations in which the policies are applied. Based on Howlett and Cashore’s 
(2009) premise that policy instruments are regarded as the core concept of policy output 
which can be designed, Knill et al. (2012) distinguished between the number of policy 
instruments deployed and the content of policy instruments, developing specific kinds of 
density and intensity measures. 

Defined merely as the cumulative number of instruments deployed in a mix (Schaffrin et al. 
2015), density is the most straightforward measure of mix complexity. However, merely arriving 
at a figure telling us how many instruments are deployed in a mix does not in itself tell us what 
the preferred level of density of a mix is. Whereas in the past, the Tinbergen maxim of one 
tool-one goal was often cited as an optimal density goal (Tinbergen 1952; del Rio and Howlett 
2013), more complex instances require us to delve into questions of tool interactions so as to 
better inform policy design practice, including the specification of optimal levels of policy tool 
density. Thus, to assess the effectiveness of a policy mix in the environmental area, for example, 
even if this just comprises price and quantity instruments, one must incorporate both synergis- 
tic and counterproductive tool relationships and interactions between these two tools (del Rio 
2010; Leplay and Thoyer 2011; Grabosky 1995) and control for procedural and spillover effects 
which might increase or decrease the number of tools and goals but in both cases beyond that 
of the Tinbergen target. 

In dealing with and developing the concept of “intensity,” Schaffrin et al. (2015) similarly 
relied on the same premise as Knill et al. (2012) regarding the need to look at both the “quality” 
and the “quantity” of tools used. To capture this qualitative aspect, they considered the content 
of the instruments deployed, focusing on their setting and calibrations. Thus, they defined 
policy intensity as the “organization and mobilization of resources” (Albrecht and Arts 2005: 
888): that is, the amount of resources, effort, or activity invested in or allocated to a specific 
policy instrument. Like Hood (1983), they argued that resources are scarce and hence govern- 
ments, ceteris paribus, would prefer to use “bureaucracy sparingly” so that the preferred level of 
intensity was that which would “get the job done” and no more. 

Tosun (2013) and Knill et al. (2012) took this insight further in operationalizing the “inten- 
sity” of policy instruments related to clean air regulation as, for example, the stringency of 
emission limits for certain pollutants (“objectives”), the specific levels of a tax or subsidy (“cali- 
brations”’), and the size of the target group of a tax (“settings”). Schaffrin et al. (2015) likewise 
identified a systematic set of intensity measures — objectives, scope, integration, budget, imple- 
mentation, and monitoring — which allowed them to weigh the different character or quality of 
a policy tool. This is similar to the analysis put forward by Attwell and Navin (2019) concerning 
the quality of the instruments deployed in the health sector to accomplish national vaccination 
programs, which, as noted earlier, focused on differences in scope, sanctions, severity, selectivity, 
and salience (Attwell and Navin 2019). 

As with “density,” however, what was an optimal level of intensity was unclear. Weighting 
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differences in the respective “intensity” of the instruments’ “calibrations” from the expectations 
of expert evaluations (e.g., Binder 1999; Coleman 1999; Mayhew 2005), for example, could 


differ widely from the preferences expressed in the media (e.g., Howell et al. 2000), reflecting 
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more popular beliefs about proper levels of resource use and misuse in specific policy areas from 
the control of crime to the provision of particular kinds of health-care services 

Thus, effective policy design requires a more careful analysis of both intensity and density 
across policy areas and levels of government (Howlett and del Rio 2015), as well as salience 
(Attwell and Navin 2019) than either measure has received to date. And this analysis should be 
undertaken beyond case- or issue-based assessments (Fernandez-i-Marin et al. 2021). This is 
especially the case surrounding additive effects and synergistic or counterproductive interactions. 
A better method of evaluating instrument mixes characteristics is needed, one that distinguishes 
between different kinds of mixes and takes into account the three different types of interactive 
tool effects discussed earlier. 


Assessing Additive, Synergetic, and Counterproductive 
Effects of Instruments in Policy Mixes 


Developing such a method begins with the recognition that to quantify the degree of synergistic 
and counterproductive effects, one must first formulate a reference model that gauges expected 
policy effectiveness when there is no interaction between policy instruments. 

This flows from the first principle in policy mix analysis, which concerns the additive nature 
of tool interactions. An additive effect is an effect that we expect to receive by simply adding 
together the impact of each individual policy instrument. This purposely does not include inter- 
action effects so that these effects may be estimated in the second stage of analysis. 

Simple addition is the basic interpretation of the measures put forward here — density and 
intensity — in which the basic nature of an instruments mix is determined by simply adding 
together the number of tools used in a mix and correlating this with the number of government 
resources utilized by each, without accounting for any other kinds of interactive effects (Oiko- 
nomou and Jepma 2008; Oikonomou et al. 2011). 

This calculation generates a minimal model for an instruments mix in which four basic 
types of mixes exist, ranging from low density-high intensity mixes (such as Tinbergen’s single 
tool deployment at the extreme) to the reverse situation in which many not-very-intense tools 
are deployed, such as in traffic control. In between, there are other possibilities such as a high 
density-high intensity mix deployed in important areas such as pandemic control and low den- 
sity-low intensity mixes, which can be found in many areas in which mainly symbolic tools are 
deployed, such as in the encouragement of healthy lifestyles through infrequent and low-cost 
public service advertising. 

Even discounting interactive effects, additive impacts are not necessarily automatic and do 
not necessarily occur all at once. Although one might expect that simply placing two policy 
instruments in a policy mix either initially or sequentially will theoretically result in additivity 
effects, in real-world applications, such additivity effects may not occur because, for instance, 
harsher policies may result in growing resistance by policy targets. A case in point is when more 
taxes lead to higher overall tax rates, giving rise to increased tax evasion and not necessarily the 
collection of more tax revenue. In other words, the relationship between the number of policy 
tools over a certain threshold and policy effects may be non-linear, and increasing the number of 
policy tools may result in lower-than-expected effects. Still, an additive measure or indicator of 
basic mix design is important because it provides a baseline and a measure of effort which can be 
used to detect and quantify synergistic or antagonistic effects. That is, any (substantial) deviation 
from simple additivity reveals synergistic or counterproductive effects. 

Synergy is commonly described in simple terms as “1+1>2.” That is, synergy occurs when 
the effect of two or more policy tools operating in a combination is greater than the (expected) 
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additive effect of the policy tools (del Rio 2010; Lecuyer and Bibas 2012). Two policy tools 
exhibit synergy when, for example, one policy tool increases the effect of the other without any 
change in its original intensity. An example of synergy is the use of both taxes and information 
campaigns to discourage smoking. Combining the two provides a greater boost in non-smoking 
than simply using one or the other because the use of anti-smoking information campaigns not 
only reinforces the effects of tax increases on discouraging smoking among smokers but also 
affects non-smokers who might otherwise consider starting. Synergistic instruments mix can 
therefore suppress policy resistance to one of the tools in the policy mix, slow the evolution of 
policy resistance, and/or facilitate lower-intensity use of each policy tool, thus reducing unin- 
tended negative consequences upon being targeted with a given instruments mix. Synergism 
in tool-tool interaction is, therefore, more than an additive effect. It is the effect of two policy 
instruments working in a combination that is greater than the (expected) additive effects of 
these instruments. This potentially enables the attainment of a higher level of policy effective- 
ness while minimizing additional policy investment. Put differently, when policy instruments 
act synergistically, it may be possible to reduce policy investment in both instruments while still 
ensuring the desired level of outcome. The identification of synergistic instruments mixes, how- 
ever, is challenging due to the infrequency of synergistic relationships. Still, synergies are more 
likely when tools have been selected in a non-political way. 

The application of mixes of policy instruments may also produce lower responses. These 
counterproductive effects can be described in simple terms as “1+1<2” or as the opposite of 
synergy. A counterproductive effect occurs when the combined impact of two tools is less than 
their additive effect. An example of such a subtractive effect can be found in the area of tobacco 
control, in which excessive taxation encourages smuggling and other forms of evasion. This 
has the effect of providing cheaper and more readily available products, undermining both the 
taxation and information provision tools used in prevention efforts. A more recent example 
in the area of the regulation of legalized cannabis products can be found in the promotion of 
responsible consumption campaigns coupled with weight- or volume-based taxation on sales. In 
some US states like Washington, this has led to large increases in the drug’s potency, offsetting or 
reducing the impact of responsible consumption campaigns (Barry and Glantz 2018). 

Such counterproductive effects are often undesirable in policy effectiveness terms. This is 
rather obvious. However, it may be beneficial to have an antagonistic effect in a mixture of 
policy tools when these effects minimize or neutralize unwanted side effects created by one of 
the policy tools in a mix. In addition to making sure that other policy instruments will not get 
out of control, antagonistic effects may be desirable for political reasons when, for example, 
elected executives use a highly visible policy tool that benefits their competitor's political base 
alongside a less visible tool which can counteract these benefits. This can occur, for example, 
when increased property taxes offset education grants to schools in specific neighborhoods 
or regions. Such cases sit squarely within the category of deliberate disproportionate policy 
responses (Maor 2017, 2021). 


Methodological Issues in Capturing Instruments Mix Superiority Over 
Individual Policy Instruments 


Understanding both these more and less than additive effects is a serious issue in policy design 
as missed synergistic and counterproductive reactions can lead to a policy over- or under- 
design (Maor 2020). And the same is true of incorrectly specified additive or subtractive 
mixes. It is crucial to improve design practice by better understanding these constructs in 
order to avoid unintentional errors and correctly search out and deploy (or not) synergies and 
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counterproductive effects. Towards this end, several methods for evaluating instruments mix 
effectiveness currently exist. 

In general, two approaches to assessing instrument combinations’ impact on outcomes can 
be advised. The first involves an “effects-based” assessment strategy. In this strategy, the idea is 
to carefully deploy policy instruments individually and sequentially in policy mixes and gauge 
their impact directly, in real time, on the kinds of outputs which emerge from their implemen- 
tation (Tupper and Doern 1981). This can be done on a small-scale experimental basis but is a 
challenging approach, given the need to control for many variables, including implementation 
barriers, capacity, and other resource issues in assessing policy impact, as well as a host of mea- 
surement problems associated with both tool deployment and output assessment. 

The second approach utilizes input efforts as a proxy for output and can be termed the “effort- 
based” approach. In this approach, measures of the effort put into deploying a policy mix by a 
government are used as a proxy for policy effects. This strategy is much easier to accomplish. But 
it may fall down precisely in not being able to assess the impact of effort on effects. In addition, it 
is capable only of assessing known additive, counterproductive, and synergistic impacts. 


Effect-Based Strategy 


The effect-based methodology is complex and requires construction of a baseline of the effects 
of individual tool deployment on goal attainment and then comparing the effects deriving from 
a mix of instruments to those stemming from the individual performance of each of the instru- 
ments involved. 

Three possible results facilitating a conclusion of positive, negative, or null effect are relevant 
here. First, the policy mix, comprised of at least two policy instruments that are individually non- 
effective, can produce a (statistically) significant effect greater than the effect produced by its indi- 
vidual components. Second, the same kind of policy mix can result in (1) an effect which is greater 
than, equal to, or less than the additive effect produced by its individual components and (2) a dif- 
ference between the two effects which also reaches statistical significance. This provides evidence 
of the superiority of the policy mix compared to the deployment of its single policy instruments. 
Third, the effect of such a policy mix can be greater than the (expected) additive effect given by the 
sum of the individual effects. This approach enables us to assess synergy by comparing the observed 
effect of a policy mix to an (expected) effect from additivity. If the observed effect of a policy mix 
is lower than the effect from additivity, it can be classified as counterproductive. 

These kinds of effect-based evaluations require careful and systematic analysis of policy effects 
over time and may not be useful in many time-delimited circumstances or when the enactment 
of multiple partial efforts at problem resolution are infeasible. However, it can be done in ongo- 
ing, slow-moving areas such as housing or the regulation of drug abuse, where partial measures 
may be introduced on an experimental or trial basis. A recent advance, in the form of an index 
that captures whether governments tend to reuse the same policy instruments and instrument 
combinations or produce policies that are tailored to the problem at hand, can assist effect-based 
evaluations by providing prescriptive statements about the extent to which the policies in a given 
sector are effective in achieving their objectives (Fernandez-i-Marin et al. 2021). 


Effort-Based Strategy 


Alternatively, one may employ a methodology which focuses on the effort that policymakers 
invest in attempting to accomplish a policy’s goals (Winter 2006; Howlett et al. 2009; Bauer and 
Knill 2014; see also Bondarouk and Mastenbroek 2018). 
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Here it is argued that policy instruments that are characterized by higher intensity have more 
effort invested in them (Schaffrin et al. 2015: 262) as are combinations that have higher density 
levels. The literature highlights five measures of intensity, three of which are resource related, 
which can be used to gauge this effort: the number of staff assigned to work on a problem 
(Hartlapp 2009; Tosun 2012; Bauer and Knill 2014; Schaffrin et al. 2015); the types of exper- 
tise, i.e., informational resources, employed to support policy implementation (Radaelli and De 
Francesco 2007; Bauer and Knill 2014; Schaffrin et al. 2015); and the financial or budgetary 
resources allocated to the implementation of policy goals (Bauer and Knill 2014; Schaftfrin et al. 
2015). The fourth aspect of effort concerns the prioritization of goals and measures within one 
policy (Winter 2006), with higher intensity implying more effort is invested in prioritizing some 
goals or measures over others with regard to the use of the policy instrument. The fifth aspect of 
effort is monitoring (May and Winter 1999; Howlett et al. 2009: 185; Beijen 2011: 159; Tosun 
2012: 442; Bauer and Knill 2014: 34; Schaftrin et al. 2015: 264), with higher intensity imply- 
ing more effort is invested in assessing the quality of policy tool performance than where such 
follow-up is poorly executed or nonexistent (Vedung 1998; Hartlapp 2009). It is also argued 
here that measuring effort may be an indication of political commitment and that effort-based 
strategy allows us to encompass “how” a policy tool is employed (e.g., how creatively policy- 
relevant information is communicated), rather than solely the amount of money invested in 
communication. Here, quantitative measures may play as important a role as quantitative ones. 

In this effort-based approach, a policy mix comprised of two policy instruments, one with a 
higher level of intensity than the other, may produce the same effect as a policy mix in which 
the former instrument has a lower intensity than the latter. The idea here is to discover at what 
intensity level for specific kinds of instruments the mix of instruments produces the greatest 
effect. As for density, the same logic applies: more dense mixes require more effort but may not 
generate greater impacts than less dense ones. This method is easily understood but still requires 
an accurate assessment of the effects of tool deployment, which is lacking in most areas. Simply 
correlating effort with some outcome variable (such as employment or carbon-reductions or 
policy adoption) remains unsatisfactory as the correlation may be spurious. 


Discussion: Conceptual and Methodological Issues in the 
Analysis of Instrument Combinations 


Based on the analysis presented here, several lessons can be derived for instruments mix research. 
First, it is essential not to confuse synergy between policy tools and policy additivity. Synergistic 
effects must go beyond additivity, but this requires accurate baseline information on individual 
tool performance to determine additive expectations against which synergistic effects can be 
measured. In addition, deviations from additivity may be caused by measurement problems 
rather than interactions between policy instruments. Thus, the larger the sample used, the more 
likely it is that such deviations represent policy-relevant effects. 

Second, it is important not to confuse synergy between policy tools and policy effectiveness. 
Whereas synergy is a measure of the kinds of interactions present between policy instruments, 
policy effectiveness is a measure of the unique result of the specific policy mix containing these 
instruments. This distinction highlights the possibility that the scope of synergy may shrink 
when one policy instrument in a mix is employed at a high intensity, and this, in turn, may result 
in a loss of effectiveness. Beyond a certain point, its independent effect may elicit near-maximal 
effectiveness, leaving little room for the other policy instrument to improve its effectiveness. 

Third, there are many issues related to the measurement of synergism and interactivity which 
need to be addressed through careful empirical study and experimentation. This is especially 
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the case with the need to examine policy mixes comprising more than two policy instruments. 
The decision on which to engage in a specific task and at what level depends on the feasibility 
of assessing the contribution of each instrument. 

Fourth, scholars and practitioners need to recognize that synergism is not solely a statistical 
issue. The idea is to gauge the possible mechanisms that underlie the observed synergy between 
policy instruments, as well as the key variables that enable us to predict features mediating 
observed or experimental synergy between policy instruments and to distinguish between weak 
and strong synergistic effects, as well as between weak and strong counterproductive effects 
(Capano et al. 2019). A data-driven approach is needed to address these questions 

Fifth, there is another whole set of issues around temporality and the time-based nature of 
some effects. Measures and methods which may work well for single-time point impacts may 
deal less well with interactions that occur over time, including the sequence in which policy 
instruments are added to a mix (Taeihagh et al. 2013). Variation in policy feedback effects may 
also play a role here. 


Conclusion: A Call for More and Better Analysis 
of Instruments Combinations 


Combining policy instruments does not guarantee a priori an increase in efficacy over a single 
policy tool. Therefore, it is desirable to identify combinations with effects greater than what is 
achieved with either policy instrument alone. Policy instrument and policy design scholars are 
consequently interested in the interaction effects when combining a policy instrument with 
other instrument(s) in a policy mix (Kern et al. 2019). Understanding the interactions among 
policy tools is vital because combinations of policy tools could be used to target a broader range 
of policy problems. Such combinations could also serve as solutions against poorly specified 
emerging and re-emerging policy problems that are resistant to individual policy instruments. 
A significant challenge exists in this area as a policy mix may be composed of interacting policy 
instruments that synergize or cancel one another’s effects (Grabosky 1995). The need to control 
for and plan synergistic and additive effects in policy mixes necessitates a search for the effective- 
ness of individual tools and their combinations: that is, assessing their contribution to the joint 
response of a policy mix. 

In this chapter, we have emphasized the need to increase the dialogue among policy scholars 
dealing with instrument combinations in different domains by advancing a common vocabulary 
and stressing the need to follow a specific methodology if the field is to advance. We discussed 
the various terms surrounding interactive effects and provided a minimal set of concepts and 
methods for their detection. We also discussed the advantages and limitations of effects- versus 
effort-based methods. Of course, this does not eliminate the need to address other questions 
related to instrument combinations but instead suggests an order to their assessment. 

There is also a great need for more comprehensive and sophisticated pre-implementation 
instruments mix analyses to guide the selection of policy instrument combinations suitable for 
implementation more successfully. Primarily, there is a need to develop methods to maximize 
synergetic tool interactions while minimizing their unintended negative consequences that take 
advantage of integrated policy development processes and their statistical and experimental anal- 
yses. Here, policy scholars and practitioners should recognize that some synergistic instrument 
combinations may be superior to other synergistic combinations in terms of policy effectiveness 
because not all synergistic effects are without cost. The search for synergistic tool interactions 
cannot be unconstrained because, theoretically, the scope of synergy may shrink when one 
policy instrument in a mix is employed at a high intensity. 
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Intriguingly, there is also a theoretical possibility that more antagonistic instrument combina- 
tions may slow the evolution of policy resistance, non-compliance, and gaming. This may occur 
in rare cases when a policy landscape is deliberately designed in a way that is difficult to compre- 
hend, thus restricting the evolution of opposition to the policy at hand as well as gaming oppor- 
tunities. For example, a policy landscape might be created by a three-instrument-combination 
in which each tool pair interacts antagonistically but the full emergent interaction between all 
the instruments in the policy mix is synergistic. 

Finally, another avenue for future research is the study of the conditions leading to dispropor- 
tionate policy responses — both over- and underreacting to problems (Maor 2012, 2014a; Maor 
et al. 2017) as well as policy bubbles (Maor 2014b, 2016, 2019) — and the role of instruments’ 
interaction effects in the creation and correction of such disproportionate responses. 
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POLICY TOOLS AND THE 
ATTRIBUTES OF EFFECTIVENESS 


Spaces, Mixes and Instruments 


M. Kerem Coban and Azad Singh Bali 


What attributes engender effective policy designs? This chapter synthesises extant literature 
on policy tools with a focus on identifying attributes that contribute to developing effective 
policy designs for mixes of policy tools. These attributes are organised across three levels: the 
macro level (coordination, coherence, consistency, goodness of fit), meso level (contingencies, 
complementarity, completeness, reversibility, transition, robustness, explicitness, resilience, 
administrative burdens), and micro level (targeting, degrees of freedom, stickiness). It is argued 
that macro-level attributes engender effective design spaces, meso-level attributes effective 
instrument mixes, and micro-level attributes effective calibrations. The chapter concludes with 
potential directions for future research to enhance our understanding of effective policy mixes. 


Introduction 


Addressing policy problems involves a careful, purposive, and informed effort to set policy 
goals and means. In this respect, single instruments can hardly address the rising complexity of 
contemporary challenges societies face. Recognising this need, the policy design scholarship has 
begun to pay more attention to instrument mixes which are “bundled or combined in a prin- 
cipled manner into policy ‘portfolios’ or ‘mixes’ in an effort to attain policy goals” (Howlett et 
al. 2015). The central question this chapter engages with is “What attributes or characteristics of 
these policy designs engender or are associated with effective policy design?” 

Design effectiveness concerns “mixing” instruments, which “can then predictably lead to 
desired policy outcomes” (Mukherjee et al. 2021), can be understood at three levels: design 
space (i.e., macro level), instrument mixes/tool portfolios (i.e., meso level), and particular types 
of instruments (i.e., micro level). The macro level refers to design space where actors inter- 
act with a multitude of stakeholders, and which structures the policy process out of which 
instrument mixes emerge. The meso level concerns effective design of instrument mixes while 
the micro level relates how these mixes and instruments are calibrated. 

This chapter adopts a similar approach to making sense of the attributes of effective policy 
design: the macro level, the meso level, and micro level. In doing so, it synthesises the existing 
literature on the attributes of effective policy designs across three levels. Macro-level (design space) 
attributes are coordination, coherence, consistency, goodness of fit (i.e., congruence). Meso-level 
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(policy mix) attributes are contingencies, complementarities, completeness, reversibility, transi- 
tion, robustness, resilience, and explicitness. Micro-level (calibrations) attributes include sticki- 
ness, flexibility (i.e., degrees of freedom), targeting, and administrative burdens. 

The rest of the chapter proceeds as follows: the next section proposes a three-level under- 
standing of effective instrument mix choice. In doing so, we revisit the attributes of effective 
instrument mixes at micro level, meso level, and macro level. The following section concludes 
with a brief reflection on potential directions for future research to enhance our understanding 
of effective instrument mix choice. 


What Are the Effective Tool Choices? 


Macro-Level Attributes: Design Space and Instrument Mix Effectiveness 


The macro level refers to design space where state and non-state actors interact through 
deliberations to tackle policy problems through appropriate and feasible instruments (Linder and 
Peters 1991). Within this space, effective instrument mixes arise on four attributes: coordination, 
coherence, consistency, and goodness of fit (congruence). These attributes interact in a manner 
that refers to the need of instrument mixes requiring coordination and integration, which 
facilitates instrument mixes being congruent with mode(s) of governance, policy regimes, and 
regime (policy) capacity. With this, instrument mixes can be inherently consistent and serve 
coherent policy goals. 


Coordination 


Policy problems are becoming more complex given inherent technical complexities and the 
cross-boundary nature of cross-field policy problems (Trein and Maggetti 2020). Address- 
ing them necessitates policy coordination (vertical and/or horizontal, Adam et al. 2019; 
Peters 2015, 2018) mechanisms and capabilities within the design space (Wegrich and Sti- 
mac 2014) because a single organisation and single instruments can hardly tackle such 
policy problems. In this context, alignment of tasks and efforts can be achieved best when 
instrument mixes are congruent with the context-specific political, policy, and institutional 
conditions. As such, interactions between individual policy instruments across levels of gov- 
ernance (Goyal and Howlett 2021) and/or across policy sectors (Mantino and Vanni 2019) 
could be established. 

Coordination serves an indispensable role for effective instrument mixes by enabling 
coherence and consistency, as coordination addresses “dispersed integrative capacity” concerns, 
in particular fragmented governance structures (Candel 2021; Ostrom 2010). Here, we know 
that diverging organisational cultures, mandates, ontological approaches to policy problems, and 
partisan politics, among many other cultural, organisational, institutional, and actor-level factors, 
constitute serious barriers for coordination (Peters 2018). Still, coordination is necessary (if not 
sufficient) to address complex, cross-boundary policy problems, which could hardly be done by 
individual actors on their own. Thus, it can facilitate policy integration that “bundles” distinct 
policy instruments and policy goals (Tosun and Lang 2017). For this, the existing research 
points to the attributes of conducive design space with an emphasis on shared, overlapping 
institutional logics across levels of governance and actors, political support to coordination, issue 
prioritisation for coordination, and existing coordination mechanisms along with coordination 
strategies (Candel 2021; Coban 2021). 
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Goodness of Fit (Congruence) 


The instrument mix congruence is a function of system-level policy capacity, policy regime coher- 
ence, and modes of governance (Hornung 2021; Howlett and Rayner 2013; Mukherjee and Bali 
2019; Peters et al. 2018; Wu et al. 2015, 2018; Woo et al. 2016). In this sense, it is broadly concerned 
with whether and to what extent instrument mixes are congruent with mode(s) of governance. 

To begin with system-level policy capacity, policy capacity is simply “the set of skills and 
resources — or competences and capabilities — necessary to perform policy functions” (Wu 
et al. 2015, 166). Policy capacity has three dimensions (Wu et al. 2018, 12): (1) system-level 
operational capacity relates to coordination and policy regime coherence; (2) system-level 
political capacity refers to trust in government, accountability, and inclusiveness of policymak- 
ing; and (3) data collection and processing capabilities, political support for policy analysis, 
and existence of policy advisory systems constitute the system-level operational capacity. Aris- 
ing on these dimensions, system-level capacity endows the design space for effective instru- 
ment mix choices. Otherwise, deficiencies in system-level policy capacity are likely to lead 
to ineffective, poor design or non-design (Howlett and Mukherjee 2014, 2018; Howlett 
and Rayner 2013). System-level policy capacity enables establishment and/or maintenance of 
coordination mechanisms among a multitude of actors and enhances coordination capacity 
(Wegrich and Stimac 2014), as well as policy analysis capabilities that are necessary for effec- 
tive instrument mixes (Capano et al. 2015). 

Furthermore, congruence addresses policy regime coherence (Jochim and May 2010; Wilson 
2000). A coherent policy regime pertains to policy ideas, institutional arrangements, and policies 
working in tandem without contradicting each other (Howlett and Rayner 2007; Hornung 2021). 

The third pillar of congruence concerns matching instrument mixes with mode(s) of gov- 
ernance (Howlett 2009; Howlett and Ramesh 2014, 2016). Instrument mix choices need to 
“take into account both the desired governance context and the actual resources available to a 
governmental [and/]or non-governmental actor in carrying out its appointed role” (Howlett 
and Rayner 2018, 395). Here, resources relate to policy capacity, as matching requires under- 
standing of which mode(s) of governance is/are suitable for particular instrument mixes (Capano 
et al. 2020). Furthermore, matching instrument mixes with relevant modes of governance might 
serve the mode of governance by “[achieving] its declared goals, and . . . [obtaining and main- 
taining] necessary political consensus among the actors” (Capano et al. 2015, 15) while address- 
ing necessary political consensus through coordination, which, in turn, can facilitate the design 
of coherent policy goals and consistent instrument mixes. 


Coherence 


Policy coherence refers to “systematic promotion of mutually reinforcing policy actions across 
government departments and agencies creating synergies toward achieving the defined objective” 
(Jones 2002, 391). Applying this understanding and building on Howlett (2017, 252), we define 
instrument mix coherence as the ability of multiple policy goals to reinforce each other and with 
instrument norms that are coordinated or integrated to achieve targeted policy objectives. 

Coherent policy goals and consistent instrument mixes are two pillars of effective instrument 
mixes (Howlett 2018, 2019a; Howlett and Rayner 2007). For coherent policy goals, Rogge 
and Reichardt (2016, 1627) suggest that there are two (interlinked) mechanisms: coordination 
and integration (see also May et al. 2005; Giest and Mukherjee 2022; Rogge 2018). As we have 
noted, coordination allows for integration and addressing fragmentation within design space, 
which, in turn, enhances policy coherence. 
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Surely, coordination and integration are not a panacea for coherence concerns insofar as we 
consider institutional, political, and other factors that influence the design process and design 
as the outcome, as Rogge (2018, 45) asserts that “complete coherence” is impossible given the 
complexity of policy problems and systems and political economic and/or socio-cultural factors, 
along with fragmentation within the design space. 


Consistency 


Instrument mix consistency is another macro-level attribute of effective instrument mixes. Con- 
sistency relates to how effectively and frictionlessly individual instruments are bundled and how 
individual instruments reinforce each other towards (coherent) policy goals (Howlett 2017, 
2019a, 2019b; Howlett and Rayner 2007; Kern and Howlett 2009; Rogge and Reichardt 2016). 

Consistent instrument mix is a function of congruence with existing or targeted mode(s) of 
governance, policy regimes, and regime (policy) capacity which facilitates coordination between 
actors towards coherent policy goals. As congruence provides the necessary institutional and 
structural environment for coordination and coherent policy goals, reinforcing interactions 
between individual instruments become more likely, while minimising the possibility of self- 
undermining instrument mixes (Jacobs and Weaver 2015) as well as avoiding inconsistent instru- 
ment mixes (Candel and Biesbroek 2016, 224). 


Meso-Level Attributes: Matching Tools for Effective Mixes 


At meso level, we discuss matching individual instruments for effective mixes and the attributes 
of such matching process. These attributes include contingencies, completeness, complementar- 
ity, robustness, resilience, transition, reversibility, and explicitness. 


Completeness 


Policy problems are mostly addressed through multiple instruments that include procedural and/ 
or substantive instruments (Bali et al. 2021a, 2021b; Hood 1983; Howlett 2000). Mixing pro- 
cedural and substantive instruments helps in designing complete instrument mixes where com- 
pleteness “means sufficient procedures supportive of the substantive goals exist in the policy 
mix” (Saguin 2019, 186). As a result, procedural instruments facilitate deliberation and coordi- 
nation, and substantive instruments, which are informed by improvements in the policy-making 
procedures, can effectively target policy goals with significant policy outcomes. 

One caveat in regards to the endeavour in targeting multiple dimensions of policy problems 
relates to policy designers falling into the over-design trap (Maor 2020) and missing the balance 
between instruments through duplicate or disproportionate instrument employment while aim- 
ing to address each and every dimension of policy problems (Constantini et al. 2017). In this 
regard, finding a balance while aiming for completeness is another concern for “optimal mixes”. 


Complementarity 


Matching instruments for effectiveness relies on understanding complementary interactions 
between bundled instruments. Here, building on the literature on institutional complementari- 
ties, one strand focuses on reinforcement (i.e., supplementation), replacement (i.e., substitution), 
and hierarchy (Crouch 2005; see also Hou and Brewer 2010). Reinforcement refers to a positive 
feedback and interaction effect without contradicting individual effect. Replacement relates to 
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one or more instruments overriding the other(s) and thereby either dominating the overall effec- 
tiveness of the mix while negating or nullifying the role and the effect of other instrument(s). 
Hierarchy considers the necessary instrument(s) being initially employed in bundling. This is 
more closely associated with the temporal dimension of instrument mix choice and which 
instruments are initially necessary (Taeihagh et al. 2013). 

The second strand emphasises additive effects, synergies, and counterproductive effects (Howlett 
2019a; Maor and Howlett 2021). Additive effects mean that bundled instruments improve the 
effectiveness of the other instruments. Synergies in instrument mixes refer to more than additive 
effects in the form of instruments generating a greater effect than their simple combination. In 
short, complementarities draw our attention to interactions between instruments in various forms 
given the above-mentioned implications for effective instrument mix choices. 


Robustness 


Robustness 


is the capacity to maintain the functions of a system (policy, political system, organiza- 

tion, or institution). 
(Capano and Woo 2017, 404, emphasis in original; see also Capano and Woo 2018; 
Howlett et al. 2018, 407; Nair and Howlett 2016, 2017) 


Robust instrument matching could be conceived as a process as well as an output of the design 
process, which allows for instrument mixes to endure uncertainties and shocks and deliver expected 
policy outcomes. Indeed, as policymakers face uncertainties and shocks, instrument mix evolves 
according to the impact of uncertainties and shocks. Thus, adaptability of instrument mixes against 
shocks and uncertainties is vital for robust instrument matching. With this, instrument mixes can 
withstand systemic or conjunctural perturbations and continue to perform policy functions and 
deliver targeted outcomes (Capano and Woo 2017, 2018; Howlett et al. 2018). 

Robustness is characterised by instrument mixes being able to counter the impact of shocks 
and uncertainties (Capano and Woo 2017). Consequently, robust instrument matching can 
achieve or maintain instrument mix effectiveness, especially when instrument mix choices are 
agile (i.e., “responding to changing public needs in an efficient way”) (Mergel et al. 2021, 
162, emphasis in original); incorporate “slack” (i.e., procedural and/or substantive redundancy 
allowing for space for manoeuvring when parts of the mix do not function) (Capano and Woo 
2018); have anticipatory (strategic) foresight (i.e., “thinking ahead in a way that things happen accord- 
ing to an expected plan, or in a way that eventual unexpected events cannot damage the expected outcome” 
(Bali et al. 2019, 4, emphasis in original); allow for diversity in design process and positive (or 
even negative) feedback mechanisms (Nair and Howlett 2016; Weaver 2010); and system-level, 
organisational, and individual policy capacities are built up while structural and institutional 
contexts allow for adaptability, room for agency, agility, and foresight, as well as precaution (Bali 
et al. 2019; Béland et al. 2020; Howlett et al. 2018). 


Resilience 


Resilient instrument matching concerns instrument mixes having the capacity to endure endog- 
enous/exogenous shocks, responding to uncertainties and fragilities, and reaching equilibrium 
while continuing to target new or previous policy goals to deliver particular policy outcomes 
(Capano and Woo 2017, 402). 
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It is necessary to note that resilience and robustness are treated separately in extant research. 
For example, Capano and Woo (2017, 2018) assert that the latter focuses on the possibility of 
retaining functionality of a policy system or instrument mixes, as in our case, whereas the for- 
mer is concerned with resisting shocks and uncertainties and “bouncing back” to equilibrium 
of various sorts. Yet the two can hardly be treated as mutually exclusive because they are closely 
interrelated. Resilient and robust instrument mixes are those that are “capable of maintaining 
the same performance in the face of any type of internal/external perturbation” (Howlett 2019a, 
270) and “quickly resume critical functions that were affected by a shock to the system [in our 
case, instrument mix]” (Boin and Lodge 2016, 293). In this light, achieving resilience is signifi- 
cantly challenging. Yet similar to robustness, it could be argued that agility, flexibility, procedural 
and/or substantive redundancy, anticipation, and foresight are necessary, if not sufficient, condi- 
tions for resilient instrument matching. 


Tiansition 


Instrument mixes might not be designed at once, as adding or removing one or more instru- 
ments is inevitable and is perhaps desirable for policy experimentation (Huitema et al. 2018) and 
effective “packages” (Howlett and Rayner 2013) rather than “patches”. Sequencing and changes 
in instrument mixes involve costs that need to be acknowledged during instrument mixing 
(Bali et al. 2019, 7; Howlett 2019b, 31). This is because transitions may originate for several 
reasons, such as unstable coalitions (Howlett and Ramesh 2002), multiple veto players (Béland 
et al. 2016), bureaucratic politics among antagonistic public organisations (Coban 2021), and 
lock-in effects of existing policies (Nair and Howlett 2016). Such factors render volatility or 
path dependencies in matching individual instruments, which might help effective mix design 
through “positive lock-in” (Howlett 2019b, 31), contributing to coherence, consistency, and 
congruence, as well as resilience and robustness, or cause “unintentional sequencing” through 
drift, replacement, or ineffective layering (Howlett 2019b, 31). 


Explicitness 


Explicitness relates to the behavioural dimension of matching instruments (Schneider and Ingram 
1990). Solving policy problems cannot ignore of the behavioural aspect of policy design: instru- 
ments need to consider and thereby match policy target motivation, preferences, and potential 
future action (Howlett 2018). For this, instrument mix explicitness is fundamental. With explic- 
itness, we refer to “the extent to which single instruments specify a desired direction of behaviour relative 
to a given policy goal” (Thomann 2018, 437, emphasis in original). 

Explicitness is based on three prerequisites Thomann (2018, 438-440). First, policymakers 
need to conceive what the problem is, how it could be addressed, and which group(s) is/are the 
relevant policy target(s) (Schneider and Ingram 1993). This stage is vital for the next two aspects 
of explicitness, as targeting “wrong” groups might create “undeserving” winners or losers or 
trigger (unexpected) negative backlash, even though it is likely that most “deserving” groups do 
not benefit as much as they need as policymakers may pursue private gains or attempt to address 
policy problems through “placebo policies” that do not tackle problems in practice (Howlett 
2020; McConnell 2020; Schneider and Ingram 2019). Still, matching individual instruments 
is crucial because different instruments may target distinct groups or may come about along 
with policy goals that are not easily reconcilable within an instrument mix. Furthermore, and 
related to framing policy goals, instrument mix choices need to take into account valence attri- 
bution, which refers to whether policy goals grab emotional attraction with which they can 
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have emotional attachment by raising awareness of the problem as well as why policy targets are 
encouraged or discouraged to comply with preferred outcomes (Olejniczak et al. 2020). 


Reversibility 


Matching individual instruments needs to take into consideration the extent to which the technical 
elements could be altered. Anticipation of such necessity is essential as contingent political, social, 
and/or financial liabilities could generate lock-in effects, which can amplify future costs, create risks, 
or generate negative feedback stemming from political backlash or various types of crises. 

In this regard, matching instruments with a focus on reversibility could give policymakers 
space to calibrate or terminate policies once they are implemented. We know that reversing or 
calibrating policies could be politically and economically costly, which can be avoided through 
“sunset” provisions (i.e., a certain termination date) or “ceilings” — targets (i.e., an upper limit 
on spending) matching other procedural and/or substantive instruments. 

In case instruments are not matched with a view toward the potential necessity to reverse or 
calibrate, political economic forces could drive incremental changes as policies accumulate and/ 
or there are abrupt calibrations or termination (Knill and Steinebach 2022). Moreover, unless 
reversal or calibration is unavoidable, an orderly resolution or termination seems more desirable. 
With an “orderly resolution”, matched instruments could be decomposed in a way that policy- 
makers can calibrate the mix as they reverse it while policy targets construct preferences about 
the path the mix would follow. 


Contingencies 


Matching individual instruments needs to take into consideration contingencies that are related 
to the attached costs of instrument mixes designed at t0 which could materialise later in tn. 
These costs relate to political liabilities as well as financial costs the instrument mix could gener- 
ate (Bali and Ramesh 2018, 336). 

Anticipating future risks and costs is demanding, and “satisficing” today (Simon 1997 [1947]) 
could be more rewarding politically and financially than matching instruments to tackle potential 
risks and costs tomorrow. However, effective instrument mix choices require engagement in 
prospective, anticipating design thinking (Bali et al. 2019). In this regard, while it is applied to public 
service delivery, O’Flynn (2019) discusses how ignoring attached liabilities could be overcome 
through a contingency framework that guides instrument mix choices through an outlook on 
targeted stakeholders and the role of stakeholders in the policy process. A contingency orientation 
in instrument mix matching can help avoid potential political backlash and/or financial costs. 

In this light, when such considerations are not included in the process of matching 
instruments, it is likely that instrument mixes might not be resilient, robust. This could, in turn, 
increase costs of calibration of instrument mixes, as policymakers might have a limited space in 
targeting, stickiness, and degrees of freedom and impose (un)intentional administrative burdens, 
which we discuss in the next section. 


Micro-Level Attributes: Calibrating Mixes for Effectiveness 


In this section, we discuss calibrating instrument mixes to maintain and/or enhance their 
effectiveness. We present targeting, degrees of freedom, stickiness, and administrative burdens in 
determining how policymakers can manage the temporal and spatial dimensions of instrument 
mix calibration for effectiveness following instrument matching and implementation. 
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Targeting 


Targeting certain populations relates to “how precisely and selectively policy instruments tar- 
get recipients of potential benefits and costs” (Landry and Varone 2005, 110). For targeting to 
identify potential benefits and costs while linking policy takers with instrument mixes, policy- 
makers engage in a reflexive, socially constructed process (Schneider and Ingram 1993). At this 
stage, policymakers grapple with two interrelated concerns: matching individual instruments 
and matching instrument mixes within the context. However, given cognitive limits, type 1 or 
type 2 errors could dominate the decision-making process, especially when policymakers face 
deep uncertainty, which could impair design effectiveness (Nair and Howlett 2016, 2017). Such 
concerns could persist following the design stage, as calibrating instrument mixes for effective- 
ness in the post-implementation stage is subject the same challenges. 

Here, Bali and Ramesh (2018, 335) note that the concern is not about whether and how 
such errors could be avoided. Calibrating mixes, while targeting addresses potential pitfalls of 
matching the wrong mixes with the context and/or policy takers, requires built-in mechanisms 
and flexibility to correct type 1 or type 2 errors and mitigate potential side effects of such errors. 
Furthermore, policymakers need to take into account the initial behavioural assumptions of each 
instrument bundled in the instrument mix, as well as the combined assumptions the mix arises 
upon and the expected behavioural outcomes it aims to produce (Schneider and Ingram 1990). 
Once policies are implemented, policy targets also make sense of the instrument mix, and they 
(non-)comply, which, in turn, determines policy performance. Respectively, calibrating instru- 
ment mixes for effectiveness relates closely to tailoring the mix both to correct potential cogni- 
tive errors and to (re)shape policy taker behaviour towards the expected outcomes. 


Stickiness 


Designing mixes that “stick” beyond the design stage and over time is a serious challenge. How 
can policymakers design instrument mixes so that “certain kinds of policy effect that gradually 
‘stick’ policies into place [are crafted], but [which] are nonetheless flexible enough to cope with 
changing external circumstances (i.e., they are not entirely ‘stuck’) [over time]”? (Jordan and 
Matt 2014, 228). Stickiness depends on policy feedback since instrument mixes create politics 
as they alter the political, social, and economic context and/or processes and their sustainability 
over time (Baekgaard et al. 2019; Béland 2010; Jordan and Moore 2020; Patashnik and Weaver 
2020; Pierson 1993). Policy feedback could be positive through increasing returns or negative 
due to weakening, degenerating forces. It drives policymakers toward stability or a change in 
instrument mixes as they adjust their preferences (Soroka and Wlezien 2009). Within this set- 
ting, policymakers need to calibrate instrument mixes to adapt to changing conditions and can 
remain intact even in different forms without losing their fiscal, social, or political sustainability 
and feasibility over time (Weaver 2010, 137). Here, it is necessary to note that stability could 
be desirable given the possibility of (too much) volatility in instrument mixes — even if not 
much is known about why and how instrument mix volatility occurs (Capano and Howlett 
2020, 8), except policymaking being captured for private gains (Howlett 2020). Still, changes in 
instrument mixes when they are subject to careful calibration might not necessarily cause drift, 
conversion, or replacement since any change could relate to adaptation. 

Noting this, scientific, political, and/or economic uncertainties can provoke “design traps”. 
Yet these could be overcome through “smart patching” and “packaging” (Nair and Howlett 
2016). Still, two interrelated major concerns can disconcert such efforts. The first relates to 
“policy bubbles” that arise upon positive feedback over a sustained period of time (Maor 2014). 
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Given entrenched interests over a long period of time, which increases returns, feeding into 
policymaker and/or policy taker expectations (Pierson 2000), policymakers can get “stuck” with 
certain mixes. Relatedly, the lock-in effects which refer to “a policy pathway that encumbers 
the implementation of policy alternatives or innovations due to high switching costs” (Popp et 
al. 2021, 2) can originate from strong feedback (Pierson 1993). Instrument mixes can behave 
like an institution (Lascoumes and Le Galés 2007) by guiding actor behaviour through the logic 
of appropriateness and/or consequences and, therefore, induce path dependence with limited 
degrees of freedom for calibration. When this occurs, instrument mix stickiness can be unde- 
sirable because stickiness can impede calibrations to improve design effectiveness (Jordan and 
Brendan 2020, 10). 


Degrees of Freedom 


Degrees of freedom determine the extent of flexibility policymakers might have. Noting this, 
degrees of freedom refer to policymakers’ capabilities to adjust instrument mixes so that instru- 
ment mixes “stick” (Jordan and Brendan 2020). Higher degrees of freedom facilitate calibrations 
for effectiveness through flexible bundling of individual instruments. As Rausser et al. (2011, 
Ch. 22) suggest, flexible instrument (mixes) allow calibrations in settings as conditions change 
and planning among policy takers regarding the settings and uncertainty. 

Here, it is necessary to note that Howlett (2018, 254) suggests that policymakers are conceived 
of having “unlimited degrees of freedom” (emphasis in original). However, such unlimited flexibility 
can rarely occur when instrument mixes are “exhausted” or “replaced”. This is because instrument 
mixes arise in particular institutional, structural, and ideational contexts wherein policymakers 
design certain mixes. Within these conditions, the degree and the extent of flexibility policymakers 
have depend on how policy regimes evolve as well as the inherent technical difficulties arising 
in time. As instrument mixes create politics, to put it differently, they constitute “instrument 
constituencies” (Béland and Howlett 2016); degrees of freedom could be narrow given entrenched 
interests, ideas, and institutional arrangements (Howlett and Rayner 2018, 397). 

Under these circumstances, instrument mix effectiveness relies on how policy regimes evolve 
and/or, as we mentioned earlier on stickiness, whether instrument mixes have built-in review, 
and auto-adjustment mechanisms. Yet while policy legacies, entrenched interests, ideas, and 
institutional arrangements define the extent and the degrees of freedom, policymakers still need 
to aim for coherence, congruence, and consistency (Howlett 2019a, 279). While doing so, poli- 
cymakers can hardly ignore administrative burdens, as they either reduce or amplify the costs of 
calibration. 


Administrative Burdens 


Administrative burden refers broadly to “an individual's [i.e., policy targets’] experience of policy 
implementation as onerous” (Burden et al. 2012). They are “hidden”, and policy targets “feel” them 
when they interact with them (Burden et al. 2012; Heinrich 2016; Herd and Moynihan 2019). 
Administrative burdens are closely related to instrument mix effectiveness and instrument 
mix calibration (Carter et al. 2018). Arguably, policy capacity deficiencies (Howlett and Ramesh 
2016) and policy non-design (Howlett and Mukherjee 2014), among other drivers, can lead to 
(in)formal (un)intentional administrative burdens. However, administrative burdens are politi- 
cally contentious (Peeters 2020); this is mainly driven by social construction of policy targets 
and interest group politics (Schneider and Ingram 1993, 1997). Deliberate policy design deci- 
sions, even if informal and unintentional, could be the product of policymakers’ preferences 
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or the involvement of powerful interest groups in the design process, which impose burdens 
on certain policy targets for rent seeking, restricting access to public services. Furthermore, 
increasing returns and positive policy feedback are likely to create entrenched interests around 
administrative burdens (Baekgaard et al. 2021). Thus, political and economic non-neutrality of 
administrative burdens means that there could be a trade-off between effectiveness and political 
economic concerns. For example, Peeters (2020) finds that policymakers who received social 
benefits before and are ideologically more pro-poor oriented are less likely to impose higher 
burdens. This finding resonates with an earlier study in which Moynihan et al. (2016) docu- 
ment that the Democratic-Republican division across states in the US is significant with respect 
to burdens policy targets face: in the states that are governed by Democratic governors, the 
Medicaid programme is less burdensome. These examples underline that besides the technical 
aspects of instrument mix calibration, managing administrative burdens can hardly ignore politi- 
cal economic concerns, as political ideologies, interest group preferences, and even personal 
experiences shape the extent of administrative burdens, which, in turn, determine policy target 
reaction that is regulated by imposed costs. 


Conclusion 


This chapter has examined attributes of effective designs focusing on the spaces in which tool 
mixes and calibrations occur. The extant literature has pointed to several attributes of good 
design in these areas, but these are rarely discussed through a three-level understanding. Address- 
ing this gap, the chapter has examined the attributes across three levels: macro level, meso level, 
and micro level. The chapter has sought to revisit and synthesise the existing research on these 
attributes. In so doing, the chapter treats the attributes separately. Yet none of them could be 
fully isolated from other remaining attributes, either on the same level or across the three levels. 
Therefore, the synthesis is not conclusive: the discussion leaves a significant and large void for us 
to fill, which offers various fruitful paths for future research on policy design, policy instruments, 
and comparative public policy. 

First, Capano and Howlett (2020) note that not much is known about effectiveness in mix- 
ing procedural and substantive instruments. Given constraints on space, the chapter could not 
adequately distinguish between or identify which types of instruments are mixed and the attri- 
butes of these mixes. Yet there are several gaps that warrant further exploration: Are we only 
examining mixes of substantive or procedural instruments? Or are we interested in the attributes 
of mixes of substantive and procedural procedures? How do instrument mixes evolve? How do 
the temporal dynamics influence the instrument mix choice, calibration, and effectiveness? Is 
there an order of mixing, and does such an order alter the effectiveness of the mix? 

Second, and related to treating the attributes separately at each level, the chapter does not 
offer a “dynamic framework” (see Bali et al. 2021, 4). A “dynamic framework” requires teasing 
out the interactions between attributes on each level and/or across levels. While the literature 
refers to the relationship between, for example, robustness and resilience or that between degrees 
of freedom and reversibility, not much is known about how individual attributes interact and 
thereby determine instrument mix effectiveness. A remedy could be studying how micro-level 
and/or meso-level attributes feed into the macro level or vice versa while endogenising the 
temporal aspects of mixing. 

Third, an implicit assumption of benign policymakers who seek to address complex, cross- 
boundary policy problems might not when policymakers pursue private gains or may have 
“hidden agendas” (Howlett 2020; McConnell 2018). More research is warranted on whether 
policymakers could instrumentalise the attributes of effective instrument mixes for private gain, 
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as instrument mixes could be effective but the distributional outcomes may favour private gains 
of certain groups. 

Fourth, we have not discussed in-depth how policy regimes influence the way and the 
process by which instrument mixes are designed. More important, we need an outlook on how 
policy regimes influence, select, or even perhaps lead volatility in instrument mixes and their 
effectiveness as policy regimes evolve. Such an approach needs to examine the role of actors, 
ideas, and the institutional and the structural context (Haelg et al. 2020). Still, this approach 
could offer limited advances to our understanding about the role of policy regimes. More could 
be achieved with endogenising political economic drivers of the policy process (John 2018). 
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CHANGING THE COMPOSITION 
OF POLICY MIXES 


Lock-in, Path Dependency and the Sequencing 
of Policy Tools 


Michael Howlett 


In recent years, work on policy design and instrument choice has advanced towards a bet- 
ter understanding of the nature of policy mixes, their dimensions, and the trade-offs between 
choices of tools, as well as the identification of basic design criteria such as coherence, consis- 
tency and congruence among policy elements. However, most of this work has ignored the 
temporal dimension of mixes or has studied this only as an important contextual variable affect- 
ing instrument choices, for example, highlighting the manner in which tools and mixes often 
evolve in unexpected or unintended ways as they age. This ignores the important issue of the 
intentional sequencing of tools as part of a mix design, either in terms of controlling spillovers 
which emerge as implementation proceeds, ratcheting up (or down) specific tool effects like 
stringency of implementation and public consultation as time passes. This chapter reviews exist- 
ing work on the unintentional sequencing of policy activity as well as the lessons which can be 
derived from the few works existing on the subject of intentional sequencing. In so doing, it 
helps define a research agenda on the subject with the expectation that this research can improve 
the resilience and robustness of policies over time. 


Introduction: Intentional and Unintentional Aspects of Temporality 
and Sequencing in Policymaking 


Policymaking is all about creating and implementing mixes of policy instruments expected to 
attain policy goals, no matter how loosely defined or publicly beneficial those goals might be. 
Given their nature, the policy tools or instruments, or the techniques used by government to 
implement policy goals (Howlett 2005), thus have a special place in considerations and studies 
of policy design (Howlett 2018, 2019a). 

Each tool has its own particular ‘character’ and exists as a bundle of attributes utilizing one 
or more governing resources (Hood 1986; Salamon 2002; Howlett 2019b). Understanding 
this character is an important aspect of designing programmes and packages likely to attain 
government goals, and, not surprisingly, many past studies have focused on identifying the 
nature, advantages and strengths and weaknesses of the varied techniques used by governments 
to implement their wishes (Salamon 2002). But choosing policy tools and designing a policy 
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become more complex when, as is very common in most policy-making situations, multiple 
goals and multiple sectors are involved in a program (Doremus 2003; Jordan et al. 2012; Howlett 
et al. 2009). In such situations, instrument choices are not restricted to a single tool, but rather 
to a mix or portfolio of policy instruments. And in constructing such bundles, balancing the 
character of different types of tools within a mix is a challenge. 

How to achieve ‘complementarity’ among mix elements, arrive at the redundancy required 
to ensure mixes are robust across space and resilient across time while, at the same time, avoiding 
excessive duplication and counterproductive relationships among tools are key questions which 
must be considered in this process (Grabosky 1995; Hou and Brewer 2010; Justen et al. 2014). 
Some combinations of tools may be purely redundant while others, even if repetitive or dupli- 
cative, may be beneficial for promoting resiliency and adaptiveness (Braathen 2005; Braathen 
2007; Swanson et al. 2010; Walker et al. 2010), and erstwhile policy designers need to be able 
to tell the difference and choose accordingly. 

Studies of policy uncertainty and policy failure, for example, have emphasized the need to 
design robustness into policies in order for them to be able to adapt to a variety of foreseeable 
futures (Moynihan 2009) and also resilience, or the ability to deal with opposition and conflict 
that may result post-enactment over the medium-to-long term (Walker et al. 2010; Kwakkel 
et al. 2010; Capano and Woo 2017). The first aspect means to design policies capable of main- 
taining the same performance in the face of any different operational context while the second 
aspect refers to the ability of the policy to adapt in the face of any type of internal/external 
perturbation (Capano and Woo 2017). Both are necessary in order to deal with surprises and 
avoid possible policy failure caused by unexpected or unknown occurrences and changing con- 
texts which upset initial design specifications and assumptions (Howlett et al. 2015a), and both 
highlight the need to investigate seriously the manner in which mixes evolve over time and how 
their various parts are and can be sequenced. 

Achieving resilience and robustness typically means duplicating some resources and 
adding procedural policy tools in order to deal with the possibility of unforeseen events 
(Howlett 2000). Studies of policy dynamics and reform (Patashnik 2008; Jordan and Matt 
2014), for example, have urged the creation of ‘sticky’ or resilient designs in long-term 
policy areas such as climate change or pensions capable of withstanding major political chal- 
lenges and performing well under changing circumstances (Jacobs 2008, 2011). This often 
means including in a policy some mechanism or procedure for monitoring and revising 
policies in a planned fashion in order to deal with future issues and spillovers from existing 
policy efforts (Lang 2019). 

Importantly, such elements of a policy need not be adopted all at once but can be sequenced 
or adopted piecemeal as a policy evolves over time. Although, in recent years, work on policy 
design and instrument choice has advanced towards a better understanding of the nature of 
policy mixes, their dimensions, and the trade-offs between choices of tools, as well as the iden- 
tification of basic design criteria such as coherence, consistency and congruence among policy 
elements, most of this work has ignored the temporal dimension of mixes. Most existing work, 
for example, has studied temporality only as an important contextual variable affecting instru- 
ment choices: for example, highlighting the manner in which tools and mixes often evolve in 
unexpected or unintended ways as they age (Howlett and Goetz 2014). 

This ignores the important issue of the intentional sequencing of tools as part of a mix design, 
either in terms of controlling spillovers which are predicted or expected to emerge as implemen- 
tation proceeds, ratcheting up (or down) specific tool effects like stringency of implementation 
and public consultation as time passes (Taeihagh et al. 2013) or establishing procedures through 
which a policy can be adapted to changing contexts (Bellehumeur 1997). 


482 


Changing the Composition of Policy Mixes 


To fill this gap, this paper reviews existing work on the unintentional sequencing of policy 
activity as well as the lessons concerning robust and resilient designs which can be derived from 
the few studies existing on the subject of intentional sequencing. In so doing, it helps move 
forward our understanding of long-term policy design and the means and mechanisms through 
which policy resilience and robustness can be achieved. 


Policy Design Issues Related to Resilience and Robustness 


Of course, how and why a policy should be designed to be robust and resilient over time is not a 
simple question, and it is well known that ensuring that policy processes, tools and outputs line 
up properly over both the short and long term is not an easy task (Jacobs and Weaver 2015). 
Attaining continued policy success over the long term is a difficult task for a number of reasons. 

As is well known, even at the best of times, there is commonly a high level of uncertainty in 
policymaking (Manski 2013), which makes anticipation of future timescapes fraught with chal- 
lenges and uncertainties. As a result, even many otherwise well-thought-out, well-intentioned 
or otherwise well-designed policies fail. Governments must grapple with complex problems 
involving situations in which they must deal with shifting coalitions of multiple actors, ideas 
and interests in complex problem environments which typically evolve and change over time. 
This so-called “VUCA’ (volatility, uncertainty, complexity, and ambiguity) world (Bennett and 
Lemoine 2014) complicates policymaking in many ways and establishes a base case for thinking 
about how such problems can be overcome. 

That is, beyond the plaints of knowledge limitations and uncertainty, governments can and 
do adopt policies with a high expectation of success, from developing road and air traffic sys- 
tems to the creation of food and drug safety regulations, all of which deal with highly uncertain 
futures and constant changes in industry actors and behaviour, public attitudes and technological 
changes, among others. Searching for the cause of failures and the means of dealing with them 
thus transcends epistemological despair as governments around the world put procedures and 
practices into place to deal with whatever levels of uncertainty exist in particular sectors and 
issue areas. 

Thus, for example, failures are known to occur due to issues relating to implementation and 
formulation practices which go beyond knowledge limits, such as a lack of resources or problems 
of vague goal definition and poor implementation, evaluation and other policy practices (Wu et 
al. 2018), not to mention the existence of corrupt or inefficient bureaucracies or other policy 
actors who are either incompetent or motivated by values other than the public good (Goodin 
1980). The presence among both state and societal actors of powerful veto players who try to 
block even the best-conceived plans — such as Obamacare in the US, for example — makes the 
process of designing effective policies over both the short and long term difficult and complex 
(Beland et al. 2016; Peters et al. 2018). 

These are also serious impediments to the very idea of ‘design’ (Turnbull 2018). But, nev- 
ertheless, the modern policy sciences were founded on the idea that accumulating and utiliz- 
ing knowledge of the effects and impacts of a relatively well-known set of policy means or 
tools developed over many years of state-building experience can help marshal and utilize the 
resources and knowledge required to overcome these obstacles and lead outcomes and processes 
towards both short- and long-term policy success (Lasswell and Lerner 1951; Howlett and 
Mukherjee 2014). 

That is, individual policies can be thought of as both more or less robust — that is, capable 
of attaining their intended effects in a variety of circumstances — and more or less resilient — or 
capable of remaining robust over time. The former concern is often centred on overcoming 
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limitations in the initial conditions when a policy is adopted while the latter addresses the issue 
that policy failures also commonly occur as the policy environment changes and evolves, often 
undermining the assumptions and expectations that went into their initial formulation (Jacobs 
and Weaver 2015; Nair and Howlett 2017; Howlett et al. 2015b). Thus, even when policies are 
designed with a clear evidentiary basis in a model formulation process so that they are robust 
and well suited to the issues and concerns of the contemporary era, they may still fail over time 
if they do not adapt to changing circumstances and concerns as policy implementation proceeds 
and the policy is put into action (Nair and Howlett 2017; Bennett and Lemoine 2014). Such 
problems affect many sectors, from health-care policies affected by long-term demographic pat- 
terns to communications and industrial policies, which are highly susceptible to technological 
change (Coleman and Basten 2015; Bastian and Börjesson 2015). 


Redundancy, Prudence and the Precautionary Principle in Policy Design 


Achieving both robustness and resilience suggests a need to be able to design and adopt poli- 
cies featuring agility and flexibility in their components and processes. However, the degree to 
which such changes can be identified and correctly anticipated at the outset of policy adoption 
varies directly with the level of ‘turbulence’ in the policy environment (Metcalfe et al. 1978; 
Salmador and Florin 2013), meaning it is not always clear a priori precisely how much agility 
or redundancy is required. 

Even in environments in which policy actors, targets and goals change only slowly and are 
relatively stable, there is always a degree of uncertainty about how long this situation will last, 
and a prudent policy, much like a bridge or a building, should always be designed within a 
relatively expansive set of risk parameters and with some degree of flexibility in the face of the 
unexpected (Capano and Woo 2017). In more turbulent circumstances, where policy ideas and 
actors change frequently (Howlett and Ramesh 2002), this problem is magnified. In such condi- 
tions, policies must be designed to be flexible. In practice, this mean policies and policymaking 
require not only additional and redundant resources but also the capability to change course as 
conditions change, such as built-in feedback mechanisms and procedures for automatic or semi- 
automatic adjustment (Soss and Moynihan 2014). 

Recognition of the need for redundancy stands in strong opposition to many ideas about 
policymaking which equate better designs with efficiency, implying that only the minimum 
possible amount of resources should be allocated to a policy, which also often emphasize routi- 
nization and the replication of standard operating procedures and program elements in order to 
ensure consistency in program delivery (Moxey et al. 1999; Cole and Grossman 1999). It also 
requires clearer thinking about what exactly the sequencing of policy elements means, how it 
occurs in policymaking and how it can best be managed to ensure resilient and robust policies 
are created and remain effective. 


Unintentional Sequencing: Historical Legacies, Layering and the 
Trajectories of Public Policies 


One critical issue around the temporality of public policymaking which policy scholars have 
begun to study concerns the trajectories of policies and their impact on the timing of the 
replacement or elimination of components and elements of policy mixes (Justen et al. 2014; 
Jordan et al. 2013; Howlett 2009). 

That is, policymaking can be viewed not as a one-off but rather as an inherently tempo- 
ral process in which policy content and outcomes shift over time, leading to patterns of the 
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sequencing of different policy elements in a trajectory of events and activities in any specific 
policy area (Mahoney 2000; Daugbjerg 2009). This statement is as true of designs intended to 
be robust and resilient as of those in which temporal considerations may have been absent in 
their initial formulation. 

The practical significance of this perspective on policy-making activities and precepts is clear 
as many older studies suggested that design and adoption occurred as processes in which policy 
packages were designed ‘de novo’ and implemented ‘en bloc’, with less attention paid to their 
evolution once in place. More recent studies, however, acknowledge that most design circum- 
stances involve building on the foundations created in another era and working within existing 
arrangements of instruments and tools put in place over an extended period of time (Howlett 
and Rayner 2013). 

Work in this area typically follows the lead of neo-institutionalists such as Thelen and Mahoney, 
specifying several processes such as layering, drift, replacement and conversion, through which 
policies have been observed to evolve (Thelen 2004; van der Heijden 2011; Béland 2007). How- 
ever, it also includes some sequential policy-making activities, such as stretching and tense layering 
(Feindt and Flynn 2009; Kay 2007), in which subsequent events and actions on the part of policy- 
makers result in a less-than-favourable pattern of policy changes as adaptations take place in a reac- 
tive way or fail to take place at all, introducing tensions into existing policy mixes which may not 
have been there when those policies were first adopted. This has happened in many sectors, such 
as land management, where subsequent incursions of new industries into formerly agricultural or 
forestry domains can result in the stretching of existing land-use policies to try to cover off the 
activities of the new industries, often with very poor results (Rayner et al. 2017). 

While focusing on the limitations of policymaking, these works have advanced our knowledge 
of temporal policy-making processes and the opportunities available to design policies capable of 
reacting to significant alterations in existing trajectories of policies and outcomes (Howlett and 
Mukherjee 2014). Their lessons and findings help us better understand policy-making processes 
and tool mix evolution and how policy portfolios can be designed to be flexible and achieve a 
higher level of resilience and robustness. How policy formulators, like software designers, can 
issue ‘patches’ in order to correct flaws in existing mixes or allow them to adapt to changing 
circumstances, for example, has been a subject of study in this vein (Howlett and Rayner 2013, 
2007), which helps identify a mechanism through which adaptations can be made to policies in 
changing circumstances in a self-reflexive way (Grin et al. 2010; Voss et al. 2006). 


Intentional Policy Sequencing: The Role of Path Dependency and 
Lock-in Effects in the Creation of Robust and Resilient Designs 


Beyond these advances, recent policy design studies have also established insights into the ques- 
tion of what makes a policy ‘sticky’, or more likely to remain in place over the long term 
(Bobrow 2006; Howlett 2011; Flanagan et al. 2011; Peters et al. 2018) and how such attributes 
can be designed into a policy at its outset. 

Path dependency, for example, is a well-known phenomenon in social processes (Arthur 
1988, 1989; David 1985, 1986; Liebowitz and Margolis 1995, 1990) and one which has been 
applied with effect in the policy sciences in order to understand the construction and mainte- 
nance of policy trajectories — that is, how initial policy actions remain more or less in effect over 
a long period of time, often being reinforced and made more difficult to change (‘locked in’) by 
the passage of time (Greener 2005). This is a common occurrence in such areas as urban hous- 
ing, for example, where initial land distributions or zoning regulations can result in patterns of 
buildings and building uses which are very difficult to change once in place. 
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Mahoney has outlined three principal elements of the general path-dependent model of 
historical evolution which characterize these circumstances. These are (1) only early events in 
sequence matter or at least matter more than ones that occur later in a temporal sequence; (2) 
these early events are said to be contingent in the sense that they could have occurred otherwise 
or not at all; and (3) later events are inertial in the sense that they follow the lead of earlier events 
and choices (Mahoney 2000). 

This view on path dependency is useful for policy design studies in two ways. First, it suggests 
that initial decisions made early in a trajectory are critical while, secondly, it also highlights that 
identifying the reasons for ‘turning points’ or ‘conjunctures’ when trajectories change is critical 
to anticipating, or guarding against, their appearance. Thus, in a policy context, for example, a 
problem in an area such as forestry around uncertain property rights which lead to overcutting, 
and trespass often leads to the creation of a rights regime such as the sale of public lands or long- 
term area-based tenures which, once in place, are very difficult to change, except around the 
edges (Moore 1957; List 2004). This lock-in, or path dependency, has important consequences 
for policy mix dynamics and the prospects for policy reform and change (Cox 2004; Dobro- 
wolsky and Saint-Martin 2005; Deeg 2001; Kay 2005) and how they can be manipulated by 
policymakers and policy designs. 

In economics, the reasons cited for lock-in include ‘network effects’, or the ability of inferior 
technologies to spread and block the adoption of more efficient ones; ‘increasing returns’, or 
the historical accident of the timing of the entry of new technologies into the marketplace; and 
(premature) ‘standardization’, which can also block the spread of superior technologies (Liebow- 
itz and Margolis 1995). These are generally negative applications of the concept. In the social 
and political realm, however, the use of the concept of path dependency is less specific and has 
been applied simply as a description of historical policy-making processes in which initial con- 
ditions matter. Hence, although often thought to be a sub-optimal affair, the fact that lock-in 
prevents changes from occurring also applies to situations in which initial policy elements can 
be put in place which lock in robust relationships and practices, thereby enhancing resilience. 


Positive Lock-in: Designing for Robustness and 
Resilience Over Time 


Much work on policy design and policy mixes has focused on the manner in which various parts 
of a mix or portfolio may be more or less integrated or coordinated (Briassoulis 2005a, 2005b). 
That is, policies are said to be composed of several elements, and some correspondence across 
these elements is required if policies are to be robust (Cashore and Howlett 2007). 

The components of a mix include policy goals and policy means at various levels of general- 
ity (Howlett 2009; Kern and Howlett 2009; Cashore and Howlett 2007), and design and instru- 
ment selection in these contexts ‘are all about constrained efforts to match goals and expectations 
both within and across categories of policy elements’ (Howlett 2009, 74). These include efforts 
to attain ‘consistency’, or the ability of multiple policy tools to reinforce rather than undermine 
each other in the pursuit of policy goals; ‘coherence’, or the ability of multiple policy goals to 
co-exist with each other and with instrument norms in a logical fashion; and ‘congruence’, or 
the ability of goals and instruments to work together in a unidirectional or mutually supportive 
fashion (Lanzalaco 2011; Howlett and Rayner 2007; Kern and Howlett 2009). 

A major issue for considerations of robustness, for example, is the fact that not all the tools 
involved and invoked in a mix may be inherently complementary (Tinbergen 1958; Grabosky 1995; 
Gunningham et al. 1998; Gunningham and Sinclair 1999; del Rio et al. 2011; Boonekamp 2006), 
in the sense that they evoke contradictory responses from policy targets, however those targets are 
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defined (Schneider and Ingram 1990a, 1990b, 1993, 1994, 1997, 2005). Such problems emerge, for 
example, in policies which simultaneously increase gasoline prices in order to discourage consump- 
tion or reduce carbon emissions and provide subsidies to automobile or truck manufacturers or buy- 
ers in order to increase sales and employment. Consumers are thus pushed and pulled in opposite 
directions by elements in the same policy portfolio (Myers 1998). A key principle of current policy 
design thinking around robustness, therefore, is to try to better integrate policy elements, maximiz- 
ing complementary or supplementary effects while minimizing counterproductive ones. 

‘Smart’ design thus implies creating policy packages which take these precepts into account in 
their formulation (Gunningham et al. 1998; Gunningham and Sinclair 1999; Eliadis et al. 2005). 
But whether or not a mix can attain, and retain, a high degree or integration or not is not just 
a matter of the character of individual tools and mixes at a single point in time but also of the 
context in which the policy evolves, which affects how these elements continue to interact and 
whether or not they continue to achieve their goals (May et al. 2005; Gilabert and Lawford- 
Smith 2012; Majone 1975). 

This attention to the ability of a policy mix to deal with changes in context is apparent in 
contemporary work aimed at better understanding and avoiding both ‘over’ and ‘under’ reac- 
tions, or how best to balance policy ‘effort’ with the severity of a problem (Haynes and Li 1993; 
Maor 2012). In most cases, with the exception of those symbolic instances where ‘over-design’ 
is welcomed, such as in areas such as national security or crime prevention (Maor 2014, 2017), 
efficient policy designs are said to be those that affect only those targets whose behaviour it is 
necessary to change and with only the minimum necessary levels of coercion and display. But 
this issue in itself varies over time as the size and type of the target can change, not least due to 
the impact of the policy itself, such as when policies devoted to building more public housing 
succeed in changing the degree of homelessness and the nature of the concerns the recently- 
housed homeless may have about their new accommodations. 

Consequently, how to enhance or alter mixes over time through the sequencing of policy 
tools so that policies are able to continue to meet old goals and take on new ones is a key ques- 
tion. Mixes that emerge over long stretches of time as a result of earlier policy decisions and 
layering, for example, often face the situation in which, even when the initial logic of a mix may 
have been clear at the outset, it can gradually transform into a degenerative or incoherent mix 
over time (van der Heijden 2011; Bode 2006; Howlett and Rayner 1995; Orren and Skowronek 
1998; Torenvlied and Akkerman 2004; Hacker 2005). 

These kinds of ‘unintentional’ mixes can be contrasted with designs which deploy sets of pro- 
cedural tools specifically intended to overcome or avoid the problems associated with layering. 
These include tools such as periodic reviews and sunset provisions, which can enhance resilience 
and robustness by building in opportunities to ‘patch’ policies and place their elements back in 
sync with each other (Gunningham et al. 1998; Kiss et al. 2013). 

The idea, of course, is to ‘lock in’ resilience within an initial design. That is, adopting mea- 
sures to ensure that the component parts of a policy mix relate well to each other not only when 
a mix is adopted but also in the future (Howlett and Rayner 2007, Grabosky 1994; Gunningham 
et al. 1998; del Rio 2010). 


Intentionally Designing Instrument Sequences to 
Ensure Resilience and Robustness: Controlling Spillovers 
and Learning From Experience 


While work on the intentional sequencing of policy elements to ensure resilience is rare, some 
work on this subject does exist and serves as a starting point for this analysis. In the mid-1970s 
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and early 1980s, for example, Bruce Doern, Richard Phidd, Seymour Wilson and others argued 
that a critical aspect of instrument choice centred on compliance and that the best way to deal 
with uncertainties around compliance involved the temporal sequencing of tools in reaction to 
compliance gains and losses. They argued that different policy instruments varied primarily in 
terms of the ‘degree of government coercion’ each instrument choice entailed (Doern 1981; 
Doern and Phidd 1983; Doern and Wilson 1974; Tupper and Doern 1981) and that tool choices 
should ‘move up the spectrum’ of coercion from minimum towards maximum if compliance 
issues persisted, and government goals failed to be met by lower-level tools. 

Preferring ‘self-regulation’ as a basic default, for example, they argued that in a good design, 
governments should first attempt to influence overall target group performance through exhor- 
tation and then add or replace instruments as and if required in order to compel recalcitrant 
societal actors to abide by their wishes, eventually culminating, if necessary, in the public provi- 
sion of goods and services. They illustrated this argument with examples of such developments 
in many countries in the air transport area, for example, where national carriers were ultimately 
created only after earlier efforts at government encouragement and subsidies failed to attract pri- 
vate sector investment in air travel (Tupper and Doern 1981). Such a vision, of course, expects 
governments to be able to avoid lock-in at sub-optimal compliance levels and to design a policy 
to be reactive to subsequent target behaviour. 

Similarly, as noted earlier, more contemporary design studies have engaged in a lengthy 
discussion as to how to better integrate policy mixes so that multiple instruments are arranged 
together in sometimes very complex portfolios of policy goals and means (Gunningham et al. 
1998; Doremus 2003; Briassoulis 2005a; Howlett 2011; Yi and Feiock 2012; Peters et al. 2005; 
Jordan et al. 2011, 2012). Work on ‘smart regulation’ by Gunningham et al. (1998) in the 
1990s, for example, led scholars and practitioners to focus on how instruments within a policy 
mix, or ‘portfolio’, could be made to complement each other (Buckman and Diesendorf 2010; 
Roch et al. 2010; Barnett and Shore 2009; Blonz et al. 2008; del Rio et al. 2010). But attain- 
ing such relationships requires certain tools to be deployed in a preparatory fashion and others 
to be deployed only after others have been put into place, as is the case, for example, with the 
initial establishment of regulations and subsidies in areas such as telecommunications and their 
subsequent revision by regulatory commissions dedicated to monitoring the environment and 
altering policy measures as conditions changed (McCraw 1975). 

Taeihagh et al. (2009, 2013) have usefully described the relationships between tools that need 
to be taken into account in initial designs (see Table 37.1). As this table shows, at least two of the 
relationships are temporal (‘precondition’ and ‘potential contradiction’) and require procedural 
tools if their deployment is to be made in such a way as to anticipate change and plan for policy 
robustness (Schrader et al. 1993; Leung et al. 2015; Bond et al. 2015). 

On a substantive level, ‘robust’ policies are those which incorporate some slack, allowing 
room for adjustments as conditions change. This is well illustrated in the case of crisis and disas- 
ter management when, in order to be able to survive crises, systems and organizations require 
redundancy, back-up systems and a greater use of materials than would normally be necessary 
for efficiency in a technical sense (Lai 2018; Cyert and March 1963). Organizations which are 
too lean (Radnor and Boaden 2004) may eliminate elements that could be useful when circum- 
stances change, thus restricting the ability of the organization to respond to surprises (Lai 2012). 

Resilience, on the other hand, requires the ability to alter and adapt policies spontaneously — to 
improvise effectively. This can involve, for example, building into a policy a range of ‘automatic 
stabilizers’ such as welfare payments or unemployment insurance payments which increase in 
the event of an economic downturn, maintaining some level of spending and saving despite 
a general economic contraction, or removing some funds from investment availability during 
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Table 37.1 Five Types of Relations Among Policy Measures 


Relation Description 


Precondition (P) Defined as a relation that is strictly required for the successful implementation of 
another policy measure. For instance, if policy measure B is a precondition to 
policy measure A, the successful implementation of policy measure A can only 
be achieved if policy measure B is successfully implemented beforehand. The 
precondition relation is a direct relation. 

Facilitation (F) In a case in which a policy measure ‘will work better’ if the outcome of another 
policy measure has been achieved, the relation is considered a facilitation relation. 
For instance, policy measure B facilitates policy measure A when policy measure 
A works better after policy measure B has been implemented; however, policy 
measure A could still be implemented independently of policy measure B. The 
facilitation relation is also a direct relation. 

Synergy (S) A special case of facilitation relation in which the ‘will work better’ relation is 
bidirectional (undirected relation). It can be argued that such a relation can be 
treated as a two-way facilitation; however, we believe that treating this relation 
as a separate type is advantageous as it suggests a higher effectiveness of both 
the policy measures having the synergetic relation vis-a-vis the overall policy. 

Potential A potential contradiction exists between policy measures if the policy measures 

contradiction (PC) produce conflicting outcomes or incentives with respect to the policy target 
under certain circumstances; hence the contradiction is ‘potential’. This 
relation is undirected. 

Contradiction (C) In contrast to the conditional nature of potential contradiction, the contradiction 
relation is defined as being when there are ‘distinctly’ conflicting outcomes 
of incentives between policy measures. Similar to the potential contradiction 
relation, this relation is undirected. 


Source: Taeihagh et al. (2013, 2009) 


boom times (Salamon 2002; Sterner 2003). But it often requires explicitly process-related tools 
to be adopted in designs, such as advisory councils in the event of regulations, whose purpose is 
to ensure future adjustments to the policy to ensure its continuing relevance. 

Policy robustness and resilience in such circumstances require the inclusion of policy-making 
procedures which allow responses to surprises to be improvised and implemented in an effective 
way as they occur (Room 2013a, 2013b). This means adding measures to existing policy portfo- 
lios that provide strategic sensitivity or the early awareness of incipient trends as they develop and 
evolve. This can include, for example, the adoption of procedural policy tools such as built-in policy 
reviews and mechanisms for outside evaluation and control, including provisions for future public 
hearings and information access, disclosure and dissemination, which allow significant adjustments 
to changing circumstances to occur (Lang 2019). This requires strategic resources or funds, people 
and competencies to be put into place that can be mobilized and (re)deployed quickly to sustain a 
policy in the face of change or, more to the point, to design policies in such a way that challenges 
might be defused before they begin (Doz and Kosonen 2014; Luthar and Cicchetti 2000). 


Conclusion: Designing for Resilience and Robustness 


In recent years, work in the policy sciences has advanced towards a better understanding of the 
nature of policy mixes and the dimensions and trade-offs that can be made between tools in a 
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superior design. A set of basic design principles has been articulated, such as the need to pro- 
mote coherence, consistency and congruence between policy elements (Howlett and Rayner 
2007; Rogge and Reichardt 2016; Rogge et al. 2017; Howlett 2018). Until recently, most of 
this work has ignored the temporal dimension of mix design, or the manner in which policies 
evolve over time, or has studied it only in terms of the unintentional sequencing of tools which 
has occurred through processes such as layering and drift (Capano 2018). However, scholars have 
now begun to turn their attention to the empirics of how policy mixes actually evolve post- 
enactment and the means by which procedural tools can aid the sequencing of policy elements 
and promote robustness, resilience and the continuing effectiveness of a policy (Howlett 2009; 
Jordan et al. 2012; Justen et al. 2014). 

This work acknowledges that policymaking is an inherently temporal process and that while lock- 
in and path dependency exist, policy content and outcomes do shift over time through processes 
such as policy patching (Kern et al. 2017). These studies seek to better understand the pressures and 
dynamics of policy components and how policies, like other artifacts, can be designed to control for 
as many events — both unforeseen and predictable — as possible, including potentially highly disruptive 
ones such as elections that bring new actors to power who may potentially undermine existing policy 
regimes and paradigms in an intentional way (Jordan and Matt 2014). 

These studies, along with those on policy uncertainty, crisis management, policy learning 
and policy capacity, have all emphasized the need to design some modicum of resilience into 
most policies, which means planning to revise them in time to deal with both expected and 
unpredictable but foreseeable future issues and spillovers from existing policy efforts (Dunlop 
2010; Howlett and Ramesh 2014; Moynihan 2009). This need for designs embodying resil- 
ience is clear. Policies that were originally thought to be ‘robust’ can become less so over time 
due to, for example, demographic changes that increase the number of elderly pension earners 
beyond original targets or due to circumstances such as climate change—inspired insect infesta- 
tions undermining projections of agricultural and forest yields. Declines in robustness also can 
occur due to policy drift or as a result of the actions of beleaguered policymakers trying to 
improve efficiencies in times of austerity, when it is hard to secure planned resource disburse- 
ments (van de Walle 2014). 

This means it is essential that policymakers are able to design policies capable of maintaining 
the same performance not only in a number of circumstances but also in the face of internal 
and external perturbations over time (Capano and Woo 2017; Howlett et al. 2015b). Policy 
designs that contain both a substantive component — a set of alternative arrangements thought 
to be potentially capable of resolving or addressing some aspect of a policy problem — as well as a 
procedural component — a set of activities related to maintaining some level of agreement among 
those charged with formulating, deciding and administering a policy — are needed to enhance 
resilience and robustness in the long term (Capano and Woo 2017; Kwakkel et al. 2010; Nair 
and Howlett 2016; OECD 2011; Walker et al. 2010, 2013). 
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Studies in the “new design orientation” have addressed many issues around policy tools and 
mixes, including how policy portfolios come to be and how they evolve, as well as the crite- 
ria for “good” policy design and a plethora of topics around policy tool use and deployment. 
Although generally following the ideas about policy compositions set out by Hall and others, 
these studies have examined the roles of policy paradigms, policy objectives, governance frames 
and general tool preferences, and policy measures and targets, but the nature and significance 
of “policy calibrations” or the “settings” of policy tools remains under-investigated and under- 
theorized. Despite suggestions that these calibrations may be crucial in driving forward paradig- 
matic change and influencing policy designs, the nature of tool calibrations has not been clearly 
established, with most existing work relying on a few works done by Ostrom in the first decade 
of this century. 


Introduction: Policy Calibrations and the Study of Policy Composition 


Studies of policy tools and mixes have addressed many issues, including how policy portfolios 
come to be and how they evolve, as well as the criteria for “good” policy design and a plethora 
of topics around policy tool use and deployment. Generally following the ideas about policy 
composition set out by Hall (1993) and Howlett and Cashore (2009), these studies have exam- 
ined the roles of topics such as policy paradigms, policy objectives, governance frames and 
general tool preferences, and policy measures and targets on policymaking and policy dynamics. 
However, the nature and significance of “policy calibrations” or the “settings” of policy tools 
highlighted by Hall in his work remain under-investigated and under-theorized. 

This is a significant gap since empirical studies suggest these calibrations are not epiphenome- 
nal but are crucial in driving forward significant change (Cashore and Howlett 2007; Oliver and 
Pemberton 2004; Siddiki et al. 2015). Most existing work relies on a few works done by Ostrom 
in the first decade of this century (Ostrom 2003, 2011), but this framework, as is discussed next, 
is inadequate in capturing the significance or range of activities characteristic of calibrations. 

This chapter re-examines this pioneering work in order to illustrate the need for a better 
understanding of policy calibrations and their impact on policy outcomes if the dynamics of 
policy reform are to be understood. 
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Policy Calibrations: The Missing Link in Policy Instrument 
and Policy Design Studies 


Peter Hall’s (1993) pathbreaking effort to disaggregate public policies provides the basis for both 
the current orthodoxy with respect to policy composition and assessments of the impact of 
changes in policy components for overall policy change. 

Hall’s work appropriately challenged the dominant view in existing scholarship that tended 
to conflate all the elements of a “policy” into a single dependent variable (Heclo 1974; Rose 
1976), a methodological decision which lay behind many early empirical accounts which sug- 
gested policy change was generally incremental in nature but could also be characterized by 
periods of more substantial change. 

Drawing on divergent cases of economic policy development in Great Britain and France, 
Hall rather argued that distinguishing between the means and ends of policymaking and between 
abstract and concrete policy decisions generated new and better insights into these processes of 
policy stability and development. As is well known, he argued that three principal elements or 
components of a policy apparent in these cases change at different rates (small scale, typical, 
incremental and larger scale, rarer, paradigmatic form) with different consequences for overall 
policy dynamics. 

For Hall, “first order” changes occurred when the calibrations of policy instruments, such as 
increasing automobile safety requirements, changed. These, he argued, occurred within exist- 
ing institutional and instrument confines and their impact on the character of overall policy 
change was minimal. “Second order” changes involved alterations to dominant types of policy 
instruments utilized within an existing policy regime, such as switching from self-regulated to 
mandatory regulations affecting automobile safety, such as seatbelt use. Such changes could be 
more substantial in terms of changing the means through which policies were implemented but 
less significant in that they continued to operate within an existing ideational framework. “Third 
order” changes, on the other hand, involved shifts in overall abstract policy goals, such as the idea 
that automobiles and traffic, generally, could be made safer than had historically been the case. 

Hall linked each change process to a different specific cause agent and to a specific overall 
pattern of what came to be termed “punctuated equilibrium” policy dynamics. In his view, first 
and second order changes were typically incremental and usually the result of activities endog- 
enous to a policy subsystem while third order changes caused paradigmatic policy alterations. 
Following Kuhn, Hall argued this latter process occurred as anomalies arose between expected 
and actual results of policy implementation, delegitimating them and allowing new ideas and 
actors to alter significant components of existing policy regimes. Further, the events triggering 
anomalies and the response to them on the part of policymakers (such as contestation within a 
policy community on the best course of action to pursue or the development of new ideas about 
policy problems and/or solutions) were commonly linked to exogenous events which affected 
social views and undermined epistemic consensus. This analysis suggested a causal sequence 
whereby significant (third order) changes in policy ideas would lead to second order changes in 
instruments which, in turn, would lead to first order changes in tool calibrations. 

Hall’s work was pathbreaking in its linking of different overall policy development processes to 
changes in the order or level of policy in flux and its generation of a testable set of hypotheses. Still, 
this initial conceptual effort at classification required some rethinking in light of its own logic, as well 
as that shed by the empirical evidence of policy change analyzed since his work was first published. 

That is, a closer examination of the components of policies revealed a more complex set 
of policy components. That is, according to Hall’s own emphasis on distinguishing abstract or 
theoretical/conceptual goals from specific programme content or objectives and operational 
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settings or calibrations and his distinction between the aims or “ends” of policy and its actual 
policy requirements (“means”), six, rather than three, policy elements exist that can undergo 
change (see Table 38.1). 

Secondly, detailed studies of long-term policy change in areas such as forestry and others sug- 
gested that significant changes were not, in fact, always driven by third order ideational changes 
but could also occur through the culminative impact of second or third order changes (Cashore 
and Howlett 2007). Empirical studies have revealed more incidences of non-paradigmatic change 
than the assumption warrants (e.g., Wilder and Howlett 2015; Hogan and Howlett 2015) as well 
as changes driven by specifications and objectives rather than abstract goals (e.g., Cashore and 
Howlett 2007). Recent empirical research into environmental and technology policy change, 
for example, suggests an alternative model whereby some significant changes occur as a result 
of instrument choices — such as the decision to create a carbon tax versus an emissions-trading 
regime in the case of climate change — or through tool calibrations — such as providing very 
generous feed-in tariffs for renewable energy deployment. 

These insights and findings have led to the need to reconsider some of the assumptions found in 
Hall’s work concerning the origins and nature of policy change processes and characteristics. As is 
set out next, recognition of several distinct change processes, among other things, highlights the role 
played by policy calibrations and the need to better understand this “missing link” in policy studies. 


Table 38.1 A Modified Taxonomy of Policy Components Following Hall 
Policy Content 
High-Level Abstraction 


Programme-Level Specific On-the- Ground 


Operationalization Measures 
Policy Ends GOALS OBJECTIVES SETTINGS 
or Aims What general What does policy What are the specific 
types of ideas formally aim to on-the-ground 
govern policy address? requirements of policy 
development? (e.g., saving wilderness (e.g., considerations 
(e.g., environmental or species habitat, about the optimal 
protection, increasing harvesting size of designated 
economic levels to create streambed riparian 
development) processing jobs) zones or sustainable 
levels of harvesting) 
Policy Focus 
Policy Means INSTRUMENT MECHANISMS CALIBRATIONS 
or Tools LOGIC What specific types What are the specific 


What general 
norms guide 
implementation 

preferences? 

(e.g., preferences for 
the use of coercive 
instruments or 
moral suasion) 


(Cells contain examples of each measure.) 


Source: Modified from Cashore and Howlett (2007) 


of instruments are 
utilized? 

(e.g., tax incentives, 
public enterprises) 
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of higher levels of 
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mandatory versus 

voluntary regulatory 


guidelines or standards) 
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An Alternative Model of Policy Change: 
Three Common Trajectories, Not One 


The “orthodox” model of policy change set out earlier based on Hall’s work postulates that 
changes in goals spur changes in tool choices and tool calibrations, which follow goal changes in 
order effect and adapt, albeit imperfectly, to the new goals. In this model, to understand policy 
dynamics, researchers thus only need to pay close attention to goal formation and change, with 
the contents of the other remaining two policy elements, tools and instrument calibrations, 
simply adjusting to follow this ideational lead (Schmidt 2008, 2010, 2011; Carstensen 2011; 
Sabatier 1988; Sabatier and Jenkins-Smith 1993; Weible et al. 2009). 

Hall’s model is set out in Table 38.2. 

But if the typical trajectory set by Hall is incorrect, is there another common pattern of 
policy development which better accords with case study evidence into the subject? Other vari- 
ants are equally plausible. One includes change through tinkering or incremental adjustment (see 
Table 38.3), in which high-level goals change and affect instrument logics, which then, in turn, 
drive changes in policy tools, specifications, and measures but which end up in a circular chain 
of specification/calibration alterations. This is the standard “incremental” or marginal model of 
policy change, the “equilibrium” pattern found in the punctuated equilibrium framework of 
policy change (Baumgartner and Jones 1993; Lindblom 1964). 

As outlined in Table 38.3, this form of policy dynamics is kicked off when, at some initial moment 
in the policy development process, goals are articulated along with general agreement about the 
nature of the means used to achieve them. In this model, “objectives” do not change but are bypassed 
by goal change—driven shifts in tool preferences and logics, which then lead to constant turmoil and 
strife around the tailoring of settings and calibrations to fit the new goals, logic, and tools. 

This is the case, for example, when governments shift preferences towards less state-centric forms 
of policy and emphasize more market ones, such as privatizations. This can lead to shifts in tools while 
objectives remain the same and then drive an extended period of tinkering with on-the-ground 
specifications of, for example, the contractual arrangements of public-private partnerships. 

Unlike Hall, however, in this formulation, the remaining aspects of policy do not simply follow 
the lead set an agreement on new general goals. In many cases, the translation of general goals 


Table 38.2 Hall’s Trajectory of Paradigmatic Change 


Policy Content 


High-Level Abstraction Operationalization On-the-Ground 
(Policy Level) (Program Level) Specification 
(Measures Level) 


Policy GOALS OBJECTIVES SPECIFICATIONS 


Ends or Aims >» 
% 3 p x 


A y a 


Policy INSTRUMENT TOOLS/INSTRUMENTS CALIBRATIONS 
Means or LOGIC What specific types of 


Tools 
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Table 38.3 Vicious Policy Development (“Tinkering’’) 


Policy Content 


High-Level Abstraction Operationalization On-the-Ground Specification 
(Policy Level) (Program Level) (Measures Level) 

Policy GOALS OBJECTIVES SPECIFICATIONS 

Ends or 

Policy INSTRUMENT TOOLS/ CALIBRATIONS 

Means or LOGIC INSTRUMENTS 


Tools What specific types of f 
\ 1 
IAJ 


into programme-level objectives, suggested by Hall to follow automatically upon paradigmatic 
ideational change, may simply never occur. 

This could be due to the existence of conflicting goals (e.g., climate mitigation and economic 
competitiveness in the case of renewable energy) that structure political divisions between poli- 
cymakers and hinder agreement on programme-level objectives and tool choices. Instead, the 
choice of the means to achieve the general goals may be tainted by these unresolved conflicts, 
leading to only narrow shifts in acceptable means and their underlying norms, such as, in the 
case of renewable energy, a logic of non-intervention in the market so as not to jeopardize eco- 
nomic competitiveness. Based on these narrow understandings, the choice of expedient policy 
tools is also narrowed down. 

As Oliver and Pemberton’s reproduction of Hall’s case study pointed out, this second pattern 
of change, in fact, provides a better fit for Hall’s own original case study of UK economic policy 
transitions brought in by the Thatcher government than does a simple model of automatic sec- 
ond and first order changes brought about by a third order change in policy goals associated with 
a shift from Keynesian to Monetarism. 

This settling for “half a loaf” or strategic decision on the part of actors to “get moving” 
despite disagreement on objectives also characterizes recent US health-care debate. In that case, 
specifications and calibrations developed to implement new goals of publicly provided health 
care ended up “locked in” to a vicious cycle of tinkering which proved only partially capable of 
instituting desired goals (Béland et al. 2020). 

A third trajectory of policy development, however, also exists and is depicted in Table 38.4. 
This third model can be observed in cases such as sustainability transitions and the practices 
around them, such as the development of the UN millennium and sustainability goals. Here, 
processes are conceptualized whereby objectives are developed either directly at the start or 
through an iterated process as settings and calibrations are reformed. 

Unlike the second model, in this third instance, agreement on objectives occurs but is not 
well linked to higher-level policy goals, and another iterative process occurs but this time involv- 
ing not just changes in calibrations but between calibrations and tool choices, objectives, and 
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Table 38.4 Virtuous Policy Development (“Patching and Learning”) 


Policy Content 


High-Level Abstraction (Policy Operationalization On-the-Ground Specification 
Level) (Program Level) (Measures Level) 

Policy GOALS OBJECTIVES SPECIFICATIONS 

Ends or 

Policy INSTRUMENT LOGIC TOOLS/ CALIBRATIONS 


Means INSTRUMENTS 
or 
Tools 


policy specifications. Such an iterated process might take the form of “packaging” or “patching” 
(Howlett and Rayner 2013; Howlett and del Rio 2015) of single policy instruments or even of 
overall policy mixes. Potentially, this leads to greater agility of policy development, more robust- 
ness of policy decisions, and avoidance of policy myopia and short-termism, as well as continual 
fighting over goals and unnecessary policy churn (problems, e.g., discussed by Monios 2016 and 
Capano and Woo 2017). 

Reconceptualizing the number and type of policy elements found in Hall’s work has serious 
consequences for policy theory and practice. (For similar models based on a similar critique of 
Hall, see Daugbjerg 1997 and Smith 2000). In particular, three implications result. First, the 
links between policy components and endogenous and exogenous sources of policy change are 
more complex than Hall suggested (Bannink and Hoogenboom 2007). Second, existing clas- 
sifications of “paradigmatic” and “incremental” policy development must be revisited so that 
we can better capture the complex interplay of change processes among the six different policy 
components (Kuhner 2007). And, thirdly, the analytical and practical lens is refocused away from 
goals and ideas towards objective and instrument choices and those related to specification and 
calibrations of those choices. 


Dealing With the Missing Link: Operationalizing and Measuring 
Policy Calibrations 


Many studies of programme-level goals and instrument choices exist (e.g., Hood, Salamon, 
et al.). However, much less is known about calibrations and specifications. 

In 1989, Linder and Peters first described eight “attributes of instruments” which they argued 
affected specific micro-level tool calibration choices. These were complexity of operation, level 
of public visibility, adaptability across uses, level of intrusiveness, relative costliness, reliance on 
markets, chances of failure, and precision of targeting (1989: 56).' This provided some insight 
into what kinds of calibration of tools might be put into place but not in any systematic way 
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Table 38.5 Ostrom’s Seven Kinds of Policy Criteria/Rules 


Boundary rules that specify how actors were to be chosen to enter or leave these positions; 

Position rules that specify a set of positions and how many actors hold each one; 

Choice rules that specify which actions are assigned to an actor in a position; 

Information rules that specify channels of communication among actors and what information must, 
may, or must not be shared; 

Scope rules that specify the outcomes that could be affected; 

Aggregation rules (such as majority or unanimity rules) that specify how the decisions of actors at a node 
are to be mapped to intermediate or final outcomes; and 

Payoff rules that specify how benefits and costs are to be distributed to actors in positions. 


Source: Elinor Ostrom, Nobel Prize Lecture 2009, “Beyond Markets and States: Polycentric Governance 
of Complex Economic Systems” 


particularly. In her work on environmental policy design, however, the Nobel Prize-winning 
political scientist Eleanor Ostrom argued that most policies articulate at least seven different 
kinds of rules and these rules — which cover everything from who is included in a policy to who 
can contest it and what kinds of payoffs and penalties are levied on targets (see Table 38.5) — can 
be thought of as describing the range of parameters which a policy or programme requires in 
order to operate: that is, their specifications and calibrations. 

Most of these rules, however, apply more to the settings or specifications of policy than to the 
calibrations of policy tools and leave open the question of what exactly constitutes such a calibration. 


Conclusion 


Every “policy” is a complex regime of ends and means-related goals (more abstract), objec- 
tives (less abstract), and settings (least abstract), and many are more complex than was sug- 
gested by Hall. In order not to be overwhelmed by this complexity and to facilitate better 
policy practice, it is necessary to more clearly disaggregate the composition of policies and 
present a clearer picture of policy dynamics than is usually found in the existing literature 
(Liefferink 2006). 

Research into how policy mixes change has seen considerable progress in recent years but 
remains underspecified (e.g., Wood 2006; Capano 2009; Baumgartner et al. 2011; Bauer et al. 
2012). Specifically, new approaches to measuring policy output have helped better conceptual- 
ize the key dependent variables in the study of policy change and advance our empirical under- 
standing of actual policy change processes. 

Some approaches — namely, those led or influenced by Ostrom — have shed light on the 
nature of policy specifications. It remains for these to be linked to policy change outcomes, but 
a significant step in that direction requires that first an equivalent study be conducted into the 
nature of policy calibrations. 


Note 


1 In his later work, however, Peters (2000) reduced this number to seven and altered their content so 
that they became directness, visibility, capital/labour intensity, automaticity or level of administration 
required, level of universality, reliance on persuasion versus enforcement, and their “forcing versus 
enabling” nature (39). 
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Past Trends and Current Directions 


Sarah Giest, Ishani Mukherjee and Michael Howlett 


The study of policy tools has been undertaken for several decades. It has isolated and examined 
many different types of tools utilized by governments over the course of history and scrutinized 
in detail how they are arranged into policy mixes or portfolios of tools. Recent developments in 
society and technology, however, have brought to the fore the possibility of using new or previ- 
ously little-used tools such as platforms, co-production, and nudges, as well as data-driven tech- 
niques, such as big data and artificial intelligence. These are added to the toolbox governments 
have at their disposal when designing policy responses to both new and old problems. Like any 
other tool, however, each of these has its strengths and weaknesses. This chapter addresses the 
promises and pitfalls of these new tools and assesses the degree to which their deployment and 
effectiveness can be understood using the kinds of typologies and concepts developed to deal 
with more traditional policy instruments. 


Introduction: The Tools Orientation in Policy Studies 


Policymaking consists of matching solutions to problems, which, in a governance context, 
means matching policy tools to the goals governments would like to achieve while in office. 
Policy alternatives which policymakers consider in the process of arriving at their decisions are 
composed of different sets or combinations of these tools. 

Policy tools are the subject of deliberation and activity at all stages of the policy-making 
process and affect both the agenda-setting and policy formulation processes as well as decision- 
making, policy implementation, and evaluation (Howlett, 2005; Howlett et al., 2020). Taken 
together, they comprise the contents of the toolbox from which governments must choose in 
building or creating public policy. Thus, as Linder and Peters (1990) noted, policy instruments 
are especially significant to policy design as they are the techniques or means through which 
states put their plans into action. 

These tools generally fall into two types. Substantive implementation instruments are those 
used to directly affect the production, distribution, and consumption of goods and services in 
society. These include such well-known tools as state enterprises, regulatory agencies, and sub- 
sidies. Procedural implementation instruments, on the other hand, are aimed at and affect policy 
processes and deliberations (Ostrom, 1986; Howlett, 2000, 2005). These tools are an important 
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part of government activities aimed at altering policy interactions within policy subsystems, 
but, as Klijn et al. (1995) put it, they typically “structure . . . the game without determining its 
outcome” (p. 441). That is, unlike substantive tools, procedural tools affect the manner in which 
policymaking and implementation unfold without predetermining the results of those activities. 

This distinction is apparent in common definitions of governing instruments, although its sig- 
nificance is sometimes overlooked. Vedung (1997), for example, defined policy instruments used 
in implementation activities as “the set of techniques by which governmental authorities wield 
their power in attempting to ensure support and effect social change” (p. 21). This definition 
thus includes both substantive tools — those Hood (1986) defined as attempting to “effect or 
detect” change in the socio-economic system — as well as procedural tools designed to “ensure 
support” for government actions. 

Substantive instruments are expected to alter some aspects of the production, distribution, 
and delivery of goods and services in society. This is a large field of action since it extends not 
only to goods and services provided or affected by markets but also well beyond them to state or 
public provision and regulation, as well as those goods and services typically provided by family, 
community, non-profit, and voluntary means (Salamon, 1989, 2002). 

The deployment of substantive instruments, for example, affects the following: 


Who produces a good or service — for example, via licensing, bureaucracy/procurement, 
or subsidies for new start-ups 

The types of goods and services produced — for example, through bans or limits or 
encouragement 

The quantity of goods or services provided — for example, via subsidies or quotas 

The quality of goods or services produced — for example, via product standards and 
warranties 

The methods of production — for example, via environmental standards or subsidies for 
modernization 

The conditions of production — for example, via health and safety standards, employment 
standards acts, minimum wage laws, and inspections 

The organization of production — for example, via unionization rules, antitrust or anti- 
combines legislation, securities legislation, or tax laws 


The consumption and distribution effects are also manifold. Some examples of these are as 
follows: 


Prices of goods and services — such as regulated taxi fares or wartime rationing 

Actual distribution of produced goods and services — affecting the location and types of 
schools or hospitals and forest tenures or leases 

Level of consumer demand for specific goods — for example, through information release, 
nutritional and dangerous goods labelling (cigarettes), export and import taxes and bans, 
and similar activities 

Level of consumer demand in general — via interest rates and monetary and fiscal policies 


Procedurally oriented implementation tools, on the other hand, affect production, consump- 
tion, and distribution processes only indirectly, if at all (Bingham et al., 2005). Rather, they 
affect the behaviour of the actors involved in formulating policies or how they are implemented. 
Just as substantive policy tools alter or affect the actions of citizens in the productive realm, so 
procedural tools affect and alter aspects of the policy-making behaviour itself (Knoke, 1993). 
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Some of the kinds of policy-making activities that can be affected by the use of procedural 
tools (Klijn et al., 1995; Goldsmith & Eggers, 2004; Klijn & Koppenjan, 2007) are as follows: 


Changing actor policy positions 

Setting down, defining, or refining actor positions 

Adding actors to policy networks 

Changing access rules for actors to governments and networks 

Influencing network formation 

Promoting network self-regulation 

Modifying system-level policy parameters (e.g., levels of market reliance) 
Changing the evaluative criteria for assessing policy outcomes, success, and failure 
Influencing the payoff structure for policy actors 

Influencing professional and other codes of conduct affecting policy actor behaviour 
Regulating inter-actor policy conflict 

Changing policy actors’ interaction procedures 

Certifying or sanctioning certain types of policy-relevant behaviours 

Changing supervisory relations between actors 


Studying Traditional Policy Tools 


Many scholars have looked at various specific kinds of tools in the past and attempted to develop 
parsimonious ways of classifying them in order to ease the burden of their analysis and deployment. 
Cushman’s (1941) study of regulatory agencies, for example, is often cited as one of the first efforts 
to systematically define the range of possible instruments which could be used in policy design. 

Other efforts around the same time originated in the post-World War II planning exercises 
and reconstruction efforts undertaken by the United Nations and the Organization for Eco- 
nomic Co-operation and Development (OECD) in Europe, which looked at a wider range 
of tools typically used in the economic realm. Key figures in this research included Nobel 
Prize-winning development economists such as Jan Tinbergen and Etienne Sadi Kirschen, who 
published studies — including, notably, Economic Policy in Our Times (Kirschen et al., 1964) — 
dealing with the instruments for economic policy they had viewed in operation in the process 
of post-war European reconstruction. One of the first inventories of instruments was Kirschen’s 
identification of well over 40 different types of implementation instruments then prevalent in 
European economic policy-making activities, ranging from public enterprises to various forms 
of government procurement, tax incentives, and subsidy schemes. 

These studies were followed by many others examining the instruments prevalent in other 
areas such as banking and foreign policy (Hermann, 1982), adding to the list of tools, such as 
interest rate determination and other monetary and fiscal tools. Lowi (1966), in particular, 
developed the insights first put forward by scholars like Cushman (1941) that governments had 
only a small number of alternative choices in any given regulatory situation, depending on the 
amount of coercion they wished to employ in that situation, suggesting that this emphasis shifted 
over time as policy targets changed. 

Further studies refined the idea of only a limited number of “governing resources” lying 
behind each tool. Hood (1986), for example, argued that governments use four resources to 
either effect changes in their environment or detect them: nodality, meaning the resource that 
existed simply by nature of the fact that governments existed at the “centre” of social and politi- 
cal networks but that can also be thought of as “information” or “knowledge”; authority; trea- 
sure; and organization (or “NATO” in Hood’s terminology). In Hood’s scheme, implementation 
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instruments are grouped together according to which of the NATO resources they most or pri- 
marily rely upon for their effectiveness, fully recognizing that most policies use some combina- 
tion of these resources in practice (Anderson, 1977; Hood, 1986). 

Hood’s taxonomy proved useful in providing a limited number of clearly differentiated cat- 
egories or types of instruments (see Table 39.1). 

All the kinds of “traditional” substantive and procedural tools (Table 39.2) enumerated by 
Salamon (2002) can be placed into these categories, and many studies have examined each type 
of tool and why it has been utilized. These include traditional — more or less command-and- 
control-oriented — “substantive” policy tools, such as public enterprises, regulatory agencies, 
subsidies, and exhortation, and their “procedural” counterparts, such as government reorgani- 
zations, reviews and inquiries, government-NGO partnerships, and stakeholder consultations 
(Klyn & Teisman, 1991; Peters & Van Nispen, 1998). 


New Directions in Tool Use and the Use of “New” Policy Tools 


If and how government preferences among tools change over time and how new tools join the 
toolbox of government are two key questions in this field that bear upon any inquiry into the 
future of policy tools. 


Patterns in the Use of Traditional Tools 


In the case of substantive tools, over the past several decades, there has been a noticeable 
movement, in many sectors, away from the use of direct government instruments and public 


Table 39.1 Hood’s 1986 Taxonomy of Substantive Policy Instruments 


Governing resource 


Nodality Authority Treasure Organization 
Principle Use Detectors Surveys Licencing Policing Record keeping 
Effectors Public information Regulation Subsidies Government agencies 
campaign 


Source: Adapted from Hood (1986) 


Note: This analysis is easily extendable to procedural tools (Howlett, 2000). 


Table 39.2 A Resource-Based Taxonomy of Procedural and Substantive Policy Instruments 


Governing Resource 


Purpose of Tool Information Authority Treasure Organization 
Substantive Public Independent Subsidies and Public enterprises 
information regulatory grants 
campaign agencies 
Procedural Official secrets | Administrative Interest Government 
acts advisory group reorganizations 
committees funding 


Source: Adapted from Howlett (2000), based on Hood (1986) 
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enterprises and towards the use of more indirect means of goods and services delivery, such as 
partnerships, special operating agencies, and quangos, among others. Public enterprises, for 
example, have grown dramatically in many countries, both in the developed world in association 
with war efforts and in developing countries as a function of decolonization and drives toward 
economic development (Howlett & Ramesh, 2020). The spread of privatization in almost every 
country between 1970 and 2000 undermined the use of this tool (Kamerman & Kahn, 1989; 
Suleiman & Waterbury, 1990). 

However, this movement should not lead us to underestimate the resilience and continued 
presence of traditional direct government tools, especially line departments, which remain the 
backbone of most policy sectors (Aucoin, 1997). Thus, in the case of public enterprises, for 
example, it must be recalled that the term privatization carries at least two different, albeit related, 
meanings (Starr, 1989, 1990a, 1990b). In one common usage, the term is sometimes inaccu- 
rately used as a shorthand reference for general efforts made in many countries in the 1980s and 
1990s to reduce the scale or scope of government. In the second sense, however, privatization 
refers only to those very specific efforts made by the state to replace organizational instruments 
based on government ownership with those based on more indirect control — like independent 
regulatory commissions — which are much more limited in character and effect and do not nec- 
essarily entail a shift towards a wholly new governance mode. 

It is true, though, that over the same period, many regulatory activities did change in charac- 
ter, having been abolished or shifted from “enforcement” to “compliance” regimes in a parallel 
process of “deregulation” (Derthick & Quirk, 1985). With respect to authoritative substantive 
instruments, the traditional direct and indirect regulatory mechanisms, which were a feature of 
implementation in many countries, were augmented throughout the 1970s and 1980s by efforts 
to promote more voluntary regulatory regimes in a wide variety of issue areas, from environ- 
mental protection to transportation and food safety. This deregulatory movement was offset in 
many jurisdictions and sectors in more recent decades, however, by return to direct or indirect 
regulation through re-regulation of areas such as telecommunications, water, and energy in 
many countries (Majone, 1997; Ramesh & Howlett, 2006). 

There have also been some interesting developments in the patterns of use found in the use of 
traditional treasury instruments in recent decades. In the area of financial tools, changes in policy 
design have been more unidirectional, with most countries seeing a cross-sectoral government- 
wide shift in recent years from an emphasis on the use of more visible subsidies — to alter the 
behaviour of firms and consumers in the marketplace — to a preference in many sectors for less 
visible forms of tax- and royalty-based expenditures. While most economic theories push for 
visible taxes and incentives in order to promote “rational” information-driven social and eco- 
nomic behaviour, the reality in most countries has been a trend towards more and more hidden 
financial tools, especially tax-based ones. These are much more difficult to trace and quantify 
and, hence, protect the disbursing governments from charges of favouritism or other market- 
distorting practices (Howard, 1993, 1997). 

And it is also now very much a matter of course for information campaigns to accompany 
all new government initiatives. Information dissemination remains relatively low cost in terms 
of financial and personnel outlays, and efforts in this area are often thought to be highly cost 
effective and, thus, preferable to other, more expensive tools (John, 2013). But compliance with 
government urgings is a major issue and, as in all advertising (Pepsi, Coke, etc.), evaluating the 
impact of these campaigns is very uncertain. Consumers may not pay attention to information 
provided by, for example, nutritional or ecolabels or may become inured to messages repeated 
too often (Howells, 2005). Effective campaigns can also take some time to get started and evoke 
any behavioural responses, while frowned-upon behaviour — such as smoking or overeating — can 
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revert back to old habits and patterns once a campaign stops. Or, where too much information 
is provided (“information overload”), the target audience may stop listening, also leading to 
diminishing returns over time (Bougherara et al., 2007). Thus, while inexpensive, comparatively 
speaking, the political risks to the government in using this tool may be high (Young, 2007). 

With respect to procedural instruments, there has also been substantial growth in the use of 
consultative forums and mechanisms in many sectors and countries over the past several decades. 
This extends from the increased use of public hearings to the increased creation (and regulation) 
of advisory committees (Brown, 1955; Smith, 1977). In many countries as well, institutional- 
ized forms of citizen involvement in policymaking have attempted to replace agenda-setting and 
policy influence by only those actors intimately involved in a project or programme (“special 
interests”) with a process in which “outsiders” as well as “insiders” could promote new and 
alternative perspectives on these issues (Pierre, 1998). 

On the procedural financial tool front, there is not a great deal of information available from 
which to judge, but a pattern in many countries and sectors has been the increased use of such 
tools over the past 30 years in an effort to enhance and control the operation of interest articula- 
tion and aggregation systems in many sectors. Many groups receive direct funding from govern- 
ments while others are funded indirectly through tax systems which allow, for example, transfer 
of funds to non-profit and charitable groups either directly or through foundations and other 
similar mechanisms (Fraussen & Halpin, 2016; Wood & Hagerman, 2010). 

There are risks involved in such activities, though, since outside funding promotes oligarchy 
or formalization in voluntary associations and can lead to discontent both among “co-opted” 
group memberships as well as from groups which do not receive funding (Lowndes & Skel- 
cher, 1998; Smith, 2005). Ideological predispositions toward “free association” in deliberative 
democratic practices, too, are jeopardized by government manipulation of interest articulation 
systems, which can lead to further difficulties for governments that engage in this practice in a 
substantial way. As a result, many such developments and tool deployments have been hidden in 
the tax system or deployed somewhat irregularly (Pal, 1995). 

Finally, in the area of information-based procedural tools, not only has the propensity for 
governments to undertake large-scale public information campaigns accelerated, as discussed 
earlier, but so has their use of devices such as surveys and other techniques for monitoring popu- 
lations and the effort to render policy processes more transparent through the use of freedom of 
information and lobbying legislation, among others. However, an earlier generation’s efforts to 
enhance information access for the public has been somewhat curtailed by the post-9/11 envi- 
ronment of enhanced security and state secrecy. 


The Emergence and Use of New Tools 


Thus, the pattern of use of traditional substantive and procedural tools has been a mixed one, 
with a general tendency towards the use of less direct government interventions, a decided 
increase in the use of information-based tools, and a corresponding decrease in direct govern- 
ment instruments. The strength of such movements has often been overstated; however, gov- 
ernments of all types continue to deploy many kinds of tools in an effort to achieve their goals. 

A key area in contemporary tools research, however, beyond monitoring how patterns in the 
use of traditional tools have evolved, concerns the analysis and inclusion in contemporary policy 
designs of new — or, at least, apparently new — kinds of tools (Hood & Margetts, 2007). 

Most notable in the present era are concerns for a better understanding of the deployment, 
design, and operation of “digital tools” that utilize, for example, artificial intelligence (AI) as well 
as other kinds of “collaborative” tools and platforms such as crowdsourcing, co-production, and 
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social media and a new set of “behavioural” instruments, including the use of defaults and other 
tools derived from the insights of research in behavioural economics. The latter include a variety 
of tools designed to affect the automatic cognitive system in humans and “nudge” them towards 
behaviours that are in their “best interests” (Thaler & Sunstein, 2009; Thaler et al., 2010). 


New Digital Tools 


Digital policy tools can be understood in two ways: tools that are themselves of a digital nature 
and tools enabling or prohibiting technology use and data collection. The former category 
includes instruments that utilize technologies and new volumes or types of data to pursue policy 
goals — for example, the use of social media platforms (Liu, 2017; Taeihagh, 2017) to orga- 
nize delivery of services through crowdsourcing and other “co-production” procedures (Pestoff, 
2006; Pestoff et al., 2006). 

This includes the digitization of the back-office work processes of bureaucrats, which is a 
powerful procedural tool. Street-level bureaucrats’ work has changed considerably because of 
digitization and automation, and many tasks traditionally conducted by bureaucrats are now 
carried out by computers (Bovens & Zouridis, 2002). Bureaucracies are further using digital 
systems to make routine decisions and, at times, replace bureaucrats altogether, not just in record 
keeping and other kinds of clerical tasks but also in decision-making and regulation, among 
others (Snellen, 2002). Numerous governmental organizations have already implemented AI 
technologies to support their processes, for example, for anomaly detection in the context of 
focusing on the identification of most likely tax evaders or criminal recidivists (Bullock, 2019). 

These tools can be labelled digitally enabled policy instruments (Clarke & Craft, 2017). 
Another way to look at digital tools is to focus on instruments that enable or prohibit certain 
technologies or data collection. This idea encompasses creating (or limiting) demand for vari- 
ous technologies through regulation and conducting and supporting R&D activities, as well as 
facilitating data-sharing practices (Taylor et al., 2009). This is also where we often see a complex 
mix or layering of instruments due to existing regulations and requirements for new or updated 
tools for data collection or sharing practices. Smart meters, as tools to change energy consump- 
tion behaviour, are a good example of such tools because they are usually deployed by private 
stakeholders (energy companies) but rely heavily on the public (data) infrastructure regulated by 
the government. If data regulation is not updated (in time), technology development might be 
delayed, limiting the nudge effect of smart meters (Giest, 2019). 

Ultimately, both understandings of digital tools are linked and add to the repertoire of tools 
available to the government to include in policy mixes. They blend into each other when, for 
example, new (social media) data is used to enhance public data used for service delivery. Here, 
governments utilize new data to potentially sharpen existing instruments while also having 
to think about how data sharing practices are regulated and how to collaborate with private 
stakeholders. 

It is often the case, however, that not entirely new instruments appear, but rather the salience 
of certain existing kinds of policy tools changes as new possibilities for mixes are created when 
more data and the possibility of enhanced collaborative efforts with private stakeholders, for 
example, is more prevalent (Hood & Margetts, 2007). Thus, the automatic prominence of 
information in the digital process changes the supply of information available to be deployed as 
part of policy design, and the nature of the information and data resources to be brokered and 
consumed by political staff, elected officials, and citizens (Craft & Howlett, 2013). 

For some instruments, digital twins of “analog” instruments are created. In the citizen- 
interaction space, this can be seen in the differences that exist between offline and online uses 
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of certain tools. In the offline environment, for example, it is recognized that participatory 
processes are often dominated by the “* 
and reasonably comfortable in public arenas” (Bryson et al., 2013, p. 29). Digital participation 


opportunities, however, open up the playing field to the less vocal members of society since the 


usual suspects, people who are easily recruited, vocal, 


setting is less immediate and confrontational and more easily accessible than offline participa- 
tion. At the same time, research points out that “internet use increasingly reflects known social, 
economic, and cultural relationships present in the offline world, including inequalities” (Van 
Deursen & Van Dik, 2014, p. 521) and thus, to a certain extent, at least has been trending 
towards a mirroring of offline consultation, albeit on a far larger scale. Thus, it is possible that 
online platforms will only strengthen the participation of people who are readily motivated to 
participate through other channels without being truly more inclusive (Clark et al., 2013; Van 
den Berg et al., 2020). A prominent idea in this setting is that of “government as a platform.” In 
the policy tools context, the potential for platform like settings is largely seen in service toolkits, 
in which different government departments tap into the same set of instruments without addi- 
tional procurement (Ansell & Miura, 2020). 


Collaborative Tools 


A second set of “new” tools exists in the large number of collaborative tools and mechanisms 
which have emerged, including the digital ones highlighted earlier. The use of tools to promote 
collaboration and co-production is another growing area which has received very little treat- 
ment in the policy instruments literature until recently. These range from using disability and 
elderly support organizations to provide services to these populations (“co-production”) as well 
as other activities such as the use of non-governmental “stewardship” councils to provide a basic 
framework of regulation (“best practices”), as occurs with groups such as the Forest Stewardship 
Council with respect to logging activities, the Marine Stewardship Council with respect to the 
fisheries, and many others (Ansell & Gash, 2008). 

Co-production emerged as a concept that emphasized citizens’ engagement in policy design 
and delivery, but its meaning has evolved to include a broader set of political-societal relations, 
including collaboration with civil society groups and non-profit organizations (Pestoff & Brand- 
sen, 2009). This broader definition of co-production, which includes individual (e.g., citizens 
and quasi-professionals) and organizational levels (citizen groups, associations, and non-profit 
organizations) of collaboration with government agencies is now common, but its implications 
for policy design and implementation are not well understood. 

Many of these “collaborative” governance arrangements have evaded detailed scrutiny from 
a tools perspective and are often prescribed without knowing exactly (1) what they are and (2) 
under what conditions they are likely to succeed or fail (Howlett & Ramesh, 2016). Although 
often promoted as solutions to many policy and governance challenges, an a priori preference 
for collaboration has little evidence of improved policy outcomes supporting it (Williams et al., 
2020). 

The use and abuse of such arrangements thus require better analysis. Unfortunately, poor 
definitions and poor theorization plague many accounts of collaboration in which otherwise 
dissimilar governance efforts are often clumped under the same rubric and their nuances and 
differences ignored. For example, Brandsen and Honingh (2016) have aimed to capture the 
range of distinctions within the concept of co-production by identifying “two variables along 
which different types of co-production can be distinguished: the extent to which citizens design 
services delivered to them and the proximity of co-production to the primary process” (p. 7), 
allowing for distinctions to be drawn based on whether communities are involved in the design 
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and/or implementation of policy. However, the authors highlight the immense diversity of 
co-production possibilities while noting that this is fundamentally driven by the diversity of 
governance contexts — meaning deployment of such tools and any evaluation of their impact are 
challenging to compare conclusively. 


Behavioural Tools 


A third set of tools which has received a great deal of attention of late is composed of those 
behavioural modifications premised on the deployment of the insights of behavioural economics 
and psychology. This is especially the case with the notion of “nudges,” which has gained much 
traction within policymaking in recent times (Dolan & Galizzi, 2015), including the idea of 
designing better “choice architectures” (Sunstein, 2014). 

Sunstein (2014) has identified ten important nudge types which combine traditional proce- 
dural and substantive tools in sometimes new ways. These include (1) default rules; (2) simplifi- 
cation; (3) use of social norms; (4) increase in ease and convenience; (5) disclosure; (6) warnings, 
graphic or otherwise; (7) pre-commitment strategies; (8) reminders; (9) eliciting implementa- 
tion intentions, and (10) informing people of the nature and consequences of their own past 
choices. 

Most of these are variations on previous efforts to alter public and individual behaviour 
through information provision, although with a decidedly less conscious or “rational” bent. 
That is, many of these efforts are geared towards the use of what has been termed “system 1” 
thinking, which is the less cognitive and more automatic or reflexive mode of thinking com- 
pared to the more conscious “system 2” mode (Kahneman, 2013). 

Underpinning this distinction is a conception that policy targets employ heuristics or mental 
“shortcuts” in their decision making based on “system 1” thinking, which may not always cor- 
respond with their individual or society’s welfare (e.g., in terms of road safety or better use of 
public transport). Instead of relying on explicit incentives or direct coercion, which appeal to 
“system 2” thinking, the use of behavioural insights as policy tools involves indirectly affecting 
the situation within which individuals make publicly relevant decisions, such that when they 
employ their familiar heuristics (system 1), the most visible or easiest decision option for policy 
targets to take is one that is more welfare promoting, whether this is realized or not. Examples 
include signs pointing to the stairs rather than the escalators in subways to promote exercise, 
or, for example, using different-coloured seats in public trains to signal commuters to relinquish 
them to those more in need (Moseley, 2020). 

Such tools have received a great deal of attention from policy and behavioural labs and have 
been deployed by many governments in recent years, albeit often with less than the expected 
impact on target behaviour, a phenomenon which has itself become the subject of research 
(Reynolds et al., 2019; Schubert, 2017; Sunstein, 2017). This literature has found that nudges 
are rarely stand-alone instruments being implemented. Instead, they are situated in a larger 
implementation context that includes existing regulations as well as potentially counteracting 
nudges by private stakeholders (Ekhardt & Wieding, 2016). 

Academics agree that more work is needed on devising supporting policy tools for nudges, 
which can help enhance cognitive and deliberative capacities (often called “boosts”), or rely on 
more reflection on the part of policy targets rather than simply responding to heuristic triggers 
(Hertwig, 2017). To address the complexity of nudges being implemented by third parties that 
potentially serve or counteract policy goals, the idea of “budging” has been increasingly promi- 
nent in the public administration and policy literature (Oliver, 2015). It captures the need to 
look at the regulatory and market structures that affect nudge implementation. 
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Conclusion: Overall Patterns and Trends in New and Old Policy 
Tool Use — Less Movement Than Meets the Eye? 


As this discussion has shown, the patterns of tool use in contemporary policy design are much 
more varied than might initially be surmised. The general picture provided here, however, in 
terms of measures of government involvement in specific tool choices and policy designs, is of 
a number of discernible shifts. As Hood et al. (1999) and Majone (1997) have argued in the 
European case, for example, “modern states are placing more emphasis on the use of authority, 
rules[,] and standard-setting, partially displacing an earlier emphasis on public ownership, public 
subsidies, and directly provided services.” 

These trends, however, are much less dramatic than is often suggested, and the same is true 
of the deployment of the three categories of “new” policy tools cited earlier. New policy tool 
use must be compatible with previously existing governance modes if they are to survive the 
formulation process and be implemented successfully. For example, while nudging has gained 
the reputation of an integral new addition to modern policy toolkits, the academic discourse 
surrounding its contribution as a distinctly novel set of policy tools has yet to find agreement on 
why some nudges “work” while others do not. 

Furthermore, this category of instruments relies significantly on the concurrent formulation 
of supporting regulatory measures and alignment with existing institutional contexts conducive 
to their enactment (Giest, 2020; Kuehnhanss, 2019; Lepenies & Małecka, 2015). Thus, as Mose- 
ley (2020) has noted, the current policy instrument scholarship is still rife with debate regarding 
the ethical dimensions of nudging, despite “flourishing research on the efficacy, public accept- 
ability, merits[,] and limitations of this approach within public policy” (p. 21). 

Similar arguments have been raised for other novel digital and collaborative categories of pol- 
icy instruments. That is, in these instances, the process of formulation also remains constrained 
by existing policy legacies and prevalent policy styles or preferences that determine the extent 
of government proclivity for adopting any tool, including new ones (Howlett & Tosun, 2018). 

The context within which contemporary policy instruments are formulated and imple- 
mented can greatly determine how radically different or novel they are (Mavrot et al., 2019). 
Thus, the literature on policy design and modern policy formulation has reiterated time and 
again how most policy choices linked with policy instruments take place through the incremen- 
tal calibration of existing policy elements or the layering of new policy instruments on existing 
toolkits (Capano, 2019; Howlett & Rayner, 2007). 

New and transformative trajectories for policy instruments are significantly bound and 
dependent on how “any new policy attempts can navigate pre-existing policies and find ways to 
create a productive layering of existing and new policies” (Schot & Steinmueller, 2018, p. 1563). 
Howlett and Rayner (2007) have further argued that the enactment of new policy programs 
rests heavily on governance processes linked with incremental policy learning, positive feedback 
mechanisms, sunk costs, and increasing returns within policy systems. This is especially the case 
as the potential “fit” or suitability of adding any new tools to an existing mix is dependent on 
maintaining coherence and consistency with active policy portfolios (Kern & Howlett, 2009). 

In addition to these more contextual constraints, there are also concerns about the capacity 
of government to understand and properly utilize novel instruments. Deploying behavioural 
and digital tools, for example, requires extensive knowledge of the existing evidence on human 
behaviour in specific contexts and how it might be changed. Digital instrument development, 
in particular, requires the allocation of resources to review available processes and integrate them 
with new tools and techniques (Dunleavy et al., 2006; Giest & Mukherjee, 2018; Mont et al., 
2014). Research increasingly shows that “governments lack the expertise to match big data to 
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draw on a broader foundation for designing some of these instruments in conjunction with 
traditional measures,” a finding that has wide-ranging effects in different policy sectors, such as 
the development of “digital welfare systems” (Giest & Mukherjee, 2018, p. 362; Yeung, 2018). 

Nevertheless, at a time of significant and visible transitions in many policy sectors, there has 
been a surge of interest in understanding the formulation of these new tools and how they can 
contribute alongside the more traditional tools in the development of innovative policy mixes 
to support government policies and achieve government goals during these transitions. Better 
understanding and designing policy portfolios to deal with these developments requires equal 
consideration for both the conditions that lead to effective implementation of traditional policy 
tools and programs, as well as more detailed knowledge of the strengths and weaknesses and 
advantages and disadvantages of the new ones. 
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COLLABORATION 
AS THE FUTURE OF 
ORGANIZATIONAL TOOLS 


Anka Kekez 


The growing intensity and complexity of challenges facing the world today — from climate 
change and pandemics to widespread inequality and radicalization — have spurred reforms of 
policy instruments across the globe. While in the case of procedural policy tools, reforms have 
induced rather comprehensive changes from bureaucracy based toward various combinations of 
soft steering tools, transformation of substantive instruments preferences has led to the expan- 
sion of organization-based subtypes to include joint efforts of public, private, and community 
actors. As a result, contemporary practice of policy implementation and public service deliv- 
ery is increasingly marked with the widespread reliance on co-production, co-management, 
relational contracting, and other organizational arrangements based on or augmented by the 
collaboration. By using an integrated policy and management lens, the chapter brings collabora- 
tive implementation tools into sharper focus and systematically reviews their scope and innate 
features as well as their capabilities and constrains to act as a solution to a wide variety of modern 
policy and governance challenges. 


Introduction 


Growing intensity and complexity of challenges facing the world today — from pandemics and 
climate crises to widespread inequality, extreme poverty, and migration — has spurred reforms 
of public policies across the globe. The need to find more effective, more inclusive but also 
more efficient and sustainable solutions to societal problems has been particularly pertinent 
with respect to policy implementation tools as they provide means which turn plans into action 
(Linder and Peters 1990; Howlett, Ramesh, and Perl 2020). Thus, over the past several decades, 
many new substantive and procedural types of tools have been created, and many of those 
already used were transformed or combined in new policy mixes. 

In the case of procedural policy tools, ways in which they affect policy-making and imple- 
mentation processes have proliferated in order to enable participation of wider sets of actors 
and softer steering of policy-relevant behaviour (Howlett, Mukherjee, and Giest 2020). While 
actors’ involvement in policy process has been fostered by means of consultative forums, advisory 
committees, and similar participatory mechanisms, coordination of policy-related activities has 
been modernised through an increased use of advanced analytical or digital devices or with the 
application of soft management mechanisms (Lang 2019; Howlett 2019; Clarke and Craft 2017). 
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The general tendency toward application of less direct government tools has also marked 
development of substantive implementation tools. The arrangements in which these tools are 
used to directly affect production, distribution, and delivery of goods and services in the society 
have, thus, expanded to include joint efforts of public, private, and community actors (Howlett, 
Mukherjee, and Giest 2020). The movement away from primarily state-based intervention 
towards tools based on resources owned by public and private, non-profit, and community 
actors has been most strikingly vivid within the organisational types of substantive instru- 
ments. Inspired by different models of public sector reforms, in-house service delivery, public 
enterprises, and other traditional organisational instruments in many policy areas have been 
offset by tools based on diverse arrangements with external providers (Howlett, Kekez, and 
Poocharoen 2017) 

Initially, starting from the 1980s and 1990s, the quest toward more effective and efficient 
solutions to social needs or a particular target group’s problems involved transition from in- 
house provision of services toward contracted delivery by external providers, inspired by New 
Public Management. After the 2000s, on the other hand, solutions proposed by the New Public 
Governance reform model more often sought the replacement of bureaucratic and contract- 
ing arrangements with tools built around cross-sector collaboration, which involves “sharing 
of information, resources, activities, and capabilities by organizations in two or more sectors 
to jointly achieve an outcome that could not be achieved by these organizations’ separately” 
(Bryson, Crosby, and Middleton Stone 2015:648). 

More recent efforts at reform in many countries have, in fact, sought to address coordi- 
nation and cooperation challenges brought by competitive market-based organisation of ser- 
vice delivery, often promoting adaptive arrangements based on or augmented by collaboration. 
As a result, contemporary theory and practice of contracted service delivery extends beyond 
zero-sum notions of state-market relations and rather involves focus on a long-term negoti- 
ated relationship between governmental contractors and external provider(s). The origins of 
such a movement lie in the recognition of the potential that collaborative forms of contracting 
hold when acting as an alternative to classic transactional contracts (Bovaird and Loeffler 2019; 
Migone 2019). 

Moreover, the benefits of a collaborative relationship between actors involved in policy 
implementation are increasingly being recognised in those policy sectors that over last decades 
proved to be resilient toward the externalisation of public services. Modernisation of traditional 
direct government tools taking place under the umbrella of reform model recently labelled the 
Neo Weberian State, hence, implies incorporation of citizens’ perspectives through the addition 
of consultative efforts to the tools’ design and delivery (Pollitt and Bouckaert 2017). Provision of 
services by the public sector organisations and their employees is nowadays increasingly enriched 
through the participation and dialogue with stakeholders, users, and their associations. 

As a result of these trends, contemporary practice of public service delivery and goods pro- 
duction is increasingly marked by a widespread reliance on arrangements based on or augmented 
with collaboration (Kekez, Howlett, and Ramesh 2018). Organisation of childcare, elderly care, 
health care, education, waste disposal, water management, and other services or goods in many 
developed and developing countries features an increasing range of mixed tools, such as consul- 
tative in-house service delivery, partnerships, commissioning, co-management, and co-produc- 
tion. Collaboration in various hybrid delivery arrangements, it is argued, enables governments to 
mobilise a wide range of actors and resources in a joint production of high-quality public value 
(Alford and O’Flynn 2012; Bovaird and Loeffler 2019; Pestoff, Brandsen, and Verschuere 2013). 

By using an integrated policy and management lens, the chapter brings collaborative imple- 
mentation tools into sharper focus and explores their potential to act as a solution to a wide 
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variety of modern policy and governance challenges. To offer a systematic overview of collabor- 
ative organisational toolbox of government, the chapter unfolds development, scope, and innate 
features of organisational policy tools by investigating the way in which these tools have been 
(re)designed the over past decades in order to include collaboration as their core or added value. 


Policy Reforms and Organisational Tools: Pathways to Collaboration 


The increased interconnectedness and complexity of modern challenges aligned with the fiscal 
and societal pressures for both more efficient and effective public services have, over the last four 
decades, placed policy reform efforts high on the political and academic agenda. Even though 
the quest to redefine the nature and the role of a public sector in many democracies in North 
America and Western Europe in particular has led to often ambiguous and incoherent reform 
paths, it sponsored three big reform models that affected the study and reality of both public 
policy and public management (Pollitt and Bouckaert 2017).! 

Acting as three general reform models, New Public Management, New Public Governance, 
and Neo-Weberian State each induced a creation of different policy tools or different arrange- 
ments of older instruments as a mean of surmounting existing policy problems and improving 
policy outcomes (Kekez 2018). Aiming for different changes of structures and processes in pub- 
lic sector functioning, these three models have induced development of distinctive organisational 
and, with them aligned, procedural policy tools. Nevertheless, development of each set of tools 
in the last two decades has been increasingly inspired by arrangements based on or augmented 
by the cross-sectoral collaboration. 

As summarised in Table 40.1, the chapter explores different reform pathways to the usage 
of collaborative policy tools by looking at the ways in which each reform model (1) aimed to 


Table 40.1 Policy Tools and Reform Models: Tools’ Potential for Collaboration 


New Public Management New Public Governance Neo-Weberian State 


Key Substantive 
Organisational 
Policy Tools 


More effective, 
efficient, and better- 
quality public service 


Outsourcing and 
contracting out, 
but also public- 
private partnerships, 


More legitimate, 
inclusive, flexible, and 
effective public service 


Partnerships, 
co-management, 
voluntary engagement, 
co-production 


Modernised state apparatus, 
more professional, 
efficient, and more 
responsive to citizens 

Provision by public sector 
organisations, public 
enterprises 


commissioning 
Dominant Competitive Public participation, Rules and input-based 
Procedural contracting, collaboration, tools accompanied by 
Tools benchmarking, and trust, and other orientation toward 
other output-based management tool achievement of results 
tools based on outcomes and toward addressing 
and shared values citizens’ needs and wishes 
Tools With Commissioning and All tools encompassed by Consultative in-house 
Collaborative PPPs based on this model service delivery 
Potential relational contracting 


Source: Adapted from Pollitt and Bouckaert (2017), Howlett, Kekez, and Poochaoren (2017), Kekez (2019) 
and Kekez, Howlett, and Ramesh (2019) 
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produce change in a public service delivery and ways in which each has affected (re)design of 
(2) substantive organisational policy tools and (3) their procedural counterparts. Based on the 
overview of the policy toolbox related to each reform model, the chapter focuses on (4) tools 
with the strongest potential to act as collaborative organisational policy tools. 


New Public Management: From Contracting Out to Commissioning 


The reform pathway for development of contracting types of implementation tools was set by 
the New Public Management (NPM) thinking in the 1980s, which sponsored a shift in the 
study and practice of public policy away from state-based solutions toward more market-centred 
ones (Osborne and Gaebler 1992; Lane 2000). Serving for several decades as a comprehensive 
model for reform of public management and policy practices, NPM introduced the whole set 
of businesslike strategies and tools that were opening the public sector to more competition and 
choice, both within and outside the government (Hood 1991; Osborne 2010). In real-life poli- 
cymaking, the competition was, to a large extent, induced with the separation of policy design 
and policy implementation, with the latter being steered away from public sector organisations 
toward entities outside the government (Pierre 2012; Alford and O’Flynn 2012). 

For the organisational implementation tools, this meant offsetting the use of state-based 
means such as state bureaucracies, public enterprises, or direct delivery of goods and services 
with market-based instruments (Dunleavy and Hood 1994; Ferlie et al. 1996; Howlett, Kekez, 
and Poocharoen 2017). Selling public enterprises or their shares to the private sector, or even 
shedding a service responsibility, was hence considered as a way to reduce the size and the cost 
of the public sector by NPM promoters. As another way to enhance efficiency, but also as a 
means to improve the effectiveness of policy implementation, NPM proposed the reduction of 
state involvement in goods and service delivery through contractual arrangements of govern- 
ment organisations (principal) with mainly private or non-profit organisations or even other 
public sector organisations. Widely known as outsourcing or contracting out, contracted production 
of public services or goods by external providers (agents) was also presented as a way to enhance 
flexibility of policy delivery and enable access to private sector technology, production capacity, 
and innovation (Hartmann and Patrickson 2000; Ponomariov and Kingsley 2008; Kettl 2010; 
Alford and O’Flynn 2012). 

Efforts toward expanding the outsourcing of public services and goods were accompanied by 
the variety of procedural tools needed to enable distribution of roles and to ensure that entities 
engaged in a contractual relation performed their roles in an useful way (Hood 1991; Savas 2000; 
Alford and O’Flynn 2012). Public procurement and tendering procedures, hence, enabled speci- 
fication of services and goods whose provision was to be contracted, as well as the selection of 
vendors. In addition, often tightly specified contracts backed by incentives and penalties allowed 
for the identification of commitments between engaged parties while the usage of performance 
measurement tools allowed for monitoring providers’ operative conduct and endorsing policy 
outputs (Brown, Potoski, and Van Slyke 2006; Dayashankar and Ramesh 2019). 

By steering the implementation toward the arm’s-length organisations or by moving it away 
from public sector organisations, NPM has stimulated the efficiency and flexibility of the public 
service but has, at the same time, demonstrated a number of accountability or market failures 
and worsened the problems of policy coordination (Petersen, Hjelmar, and Vrangbek 2018; Bel, 
Fageda, and Warner 2010; Vigoda 2002). In tackling some of undesired effects, NPM evolved 
over time into a reform model that encompassed strategies of privatisation embedded in the 
regulation designed to protect public value. Moreover, since the 1990s, it has also included strat- 
egies in which a scope of contracting practices was expanded so as to create more cooperation 
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between engaged parties (De Bruijn and Dicke 2006; Bell and Hindmoor 2009; Sanderson et al. 
2018). 

A move toward cooperation was initiated with the public-private partnerships (PPP), an 
organisational implementation tool based on the use of private sector resources in designing, 
constructing, and maintaining tasks most often associated with infrastructure projects or specific 
services such as refuse collection or vocational training (Bovaird 2004; Alford and O’Flynn 
2012). While presented in the mid-1990s as a kinder, softer alternative to privatisation, it rarely 
involved the sharing of resources and joint efforts by business and government in a production of 
a value-added outcome (Hodge and Greve 2005). One example of such collaborative partner- 
ships is a vocational education that produces a valuable good for the business community and 
individual employers and, hence, relies on a synergy between the traditional classroom learn- 
ing provided by the public sector and the hands-on training often organised by private sector 
employers (Gray 1996; Schaeffer and Loveridge 2002). In many other examples, including a 
variety of infrastructure projects known as private finance initiatives, PPPs involve production 
that is not shared but mainly performed by the private actor. The latter often takes place under 
a complex and long-term but still classical transactional type of contracts (Alford and O’Flynn 
2012). 

A more substantive move toward deeper connection between stakeholders, service beneficia- 
ries, external providers, and public sector organisations was introduced in the 2000s with the con- 
tracting approach known as the strategic commissioning of public services (Bovaird 2016). This 
organisational implementation tool, particularly prominent in the UK and Australia, involves 
engagement of public sector organisations in an outcome-oriented acquisition of public services 
that may be produced by businesses, non-profits, or public organisations. Strategic orientation 
in commissioning implies that decisions on what is to be contracted are made upon assessment 
of public needs and outcomes (Sturgess 2018; Migone 2019). In tackling identified needs and 
reaching desired outcomes, commissioning tends to harvest the “potential that exists in engaging 
with the broad spectrum of providers and users” (Migone 2019:85). 

Strategic and collaborative orientations of commissioning have two implications for the pro- 
cedural tools government uses to govern its design and implementation. Firstly, rather than 
mandating activities and prescribing detailed inputs and short-term outputs, commissioners 
steward the public service market through identification of outcomes or high-level outputs 
(Migone 2019; Sturgess 2018). Secondly, and for this chapter, more importantly, unlike classic 
contracting-out practices that are based on a traditional transaction-based relationship between 
actors, commissioning favours a more active engagement of involved actors and relies on a 
collaboration-based scope of contracting (Migone 2019; Bovaird and Loeffler 2019). This often 
implies the usage of so-called relational types of contracts involving long-term negotiated con- 
tracting based on norms of reciprocity, trust, discretion, information exchange, and jointness. 
Performance improvement in such a contract, promoters claim, is grounded not primary in 
monitoring, incentives, and sanctions but mainly on joint investments in a good relationship 
manifested in mutual support and active engagement (Van Slyke 2009; Dehoog 1990; Alford 
and O’Flynn 2012) 

Involvement of actors in a collaborative commissioning can include individual commission- 
ers and providers but can also imply arrangements such as the prime contracting or joint com- 
missioning. Prime contracting relates to examples common in public sector construction in 
which one commissioner transfers to a prime provider a whole set of (sub)contracting tasks and 
decisions on activities needed to reach the outcome. Joint commissioning, on the other hand, 
includes clustering local authorities or other types of public organisations and their joint engage- 
ment with a single or partnership of providers. Integration of services, with examples from the 
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UK health and social care system, is used to ensure “joined up” design and delivery of public 
service (Bovaird and Loeffler 2019; Sturgess 2018; Bovaird, Dickinson, and Allen 2012). 

Relational contracts between public and private entities, prime contracting, and joint com- 
missioning all add layers of complexity to contracting practices in public service delivery and, 
hence, require governments that are incentivized and capable of employing sufficient political, 
analytical, and managerial capacity (Bovaird and Loeffler 2019; Kekez, Howlett, and Ramesh 
2019). In addition, the success of this set of emerging implementation tools relies on the ability 
of public managers to “balance the returns from building a cooperative relationship with a single 
vendor against the continued risk of opportunism and the perception that the vendor's long- 
term relationship stems from political favouritism” (Brown, Potoski, and Van Slyke 2006:328). 
Even though it carries high transaction costs in the short term, when employed with the match- 
ing of types of services and managed with adequate competences, collaborative commissioning 
holds the potential to serve as an implementation tool that induces innovation in public services 
and generates strategic benefits for engaged actors (Migone 2019). 


New Public Governance: Co-managing and Co-producing Public Service 


Originating in the 1990s from the network approach to policy design and management, New 
Public Governance (NPG) as a second reform model pushed the reform agenda toward more 
reciprocal and horizontal inclusion of external actors in the policy process. Aiming to make 
government doings more legitimate, flexible, and effective, NPG proposed the idea of working 
in partnerships toward producing joint results with the stress on citizens’ participation in both 
policymaking and delivery (Koppenjan and Klijn 2004; Osborne 2010; Kekez 2019). Reliance 
on partnerships, it is considered, induces knowledge sharing throughout the policy process 
while co-production with citizens and their associations enables an injection of citizens’ values 
and priorities in the design and delivery of public services (Ostrom 1996; Pestoff and Brandsen 
2013; Kekez 2018). 

For the organisation-based substantive policy tools, this implied conscious strategy toward 
arrangements based on cross-sectoral collaboration or enhanced engagement of citizens and 
their associations in goods and service delivery. In such partnering arrangements, the key role 
in the production of public services is often given to civil society actors: namely, non-profit 
organisations or community organisations (Brandsen and Pestoff 2006). In the arrangement 
increasingly known as co-management, civil society organisations are mandated to produce pub- 
lic service such as shelters for victims of domestic violence or home care services for the elderly 
or to govern common or public good, such as housing, arts and culture, renewable energy, or 
water utilities. In taking on their managing role, nevertheless, civil society organisations do not 
act alone, but the decisions on how service is designed, how delivery is managed, and benefits 
from services or goods are made in partnership with public sector organisations, businesses, and 
society actors who can be engaged in the implementation process (Brandsen and Hout 2006; 
Steurer 2013). In that sense, co-management as the organisational tool implies collaborative 
interaction and joint contribution of multiple actors to the provision of public services and their 
outcomes (Kekez, Howlett, and Ramesh 2018). 

In the policy practice, co-management is very often combined with volunteer engagement, 
as well as with co-production, another collaborative organisational tool which involves active 
contribution by individual citizens to a service they personally receive (Alford 2009; Voor- 
berg and Meerkerk 2019). Engagement of citizens in design and delivery of a professionally 
produced service implies that, via their inputs, citizens may not only contribute to a service 
which they personally receive but also perform activities complementary to the core mission 
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of the service-providing organisation (Brandsen and Honingh 2016). While citizens’ contribu- 
tions to services need to be made in close interaction with the employees providing the service, 
it is widely argued that when users act as experiential experts, the service has more chances to 
effectively tackle the real needs of individuals, and service provision has more likelihood to be 
accompanied by the development of user accountability (Bovaird 2007; Ostrom 1996). 

Employment of co-management and co-production as organisational tools implies that deliv- 
ery of services and goods is dependent on cooperation between governmental and social actors, 
be they individual citizens or their associations. Incentivising citizens’ engagement and holding 
actors in collaborative forms of coordination are possible only if the implementation of co- 
production is based on process-oriented tools though which government enables a meaningful 
participation of societal actors in the service delivery process (Hill and Hupe 2014; Howlett, 
Kekez, and Poocharoen 2017). Those tools include recruitment of participants, creation and 
maintenance of network of service providers, and development of ways to govern the opera- 
tional complexity that comes with collaborative service delivery (Agranoff and McGuire 2004, 
Kekez 2019). 

To ensure intrinsic cooperation and participation, instead of the adoption of rules or speci- 
fication of outputs, NPG favours trust, shared values, and other soft types of procedural tools 
(Hansen 2007; Hill and Hupe 2014; Klijn 2012; Agranoff 2007). While managerial reliance 
on trust and shared responsibility, it is argued, discourages competitiveness and hostility among 
participating organisations, reliance on implementation partnerships can enhance knowledge 
sharing and mobilise unutilised resources (Howlett, Kekez, and Poocharoen 2017; Poocharoen 
and Ting 2015; Sorensen 2012). Even though the design and implementation of such complex 
procedural tools requires a high level of managing competencies for both public managers and 
street-level workers, co-management and co-production are increasingly recognized as tools 
enabling successful provision of complex services requiring high levels of creativity. 


Neo-Weberian State: In-House Delivery Modernized With Consultations 


NPM and NPG are often presented as two alternative general reform models in scholarly discus- 
sions. Many country- or area-specific models originating from empirical research or theoretical 
innovation are often subsumed under or correlated with one of these two conceptual hubs. 
However, as the comparative research on reform results shows, reforms in many Northern and 
Central European countries can hardly be correlated with paradigms behind either of the two 
big models (Pollitt and Bouckaert 2017). These countries have been proven to follow a mod- 
ernisation path which does not imply distancing the public sector from the bureaucratic nature 
of governance and the hierarchy of the traditional model of bureaucratic power. Even though 
continental modernisers have sometimes been qualified merely as guardians of the “‘status quo” 
or as stragglers reluctant to abandon the notion of the “old” hierarchy, Pollitt and Bouckaert 
(2017) see their path as a distinctive reform model and label it the Neo-Weberian State (NWS). 

The label itself expresses the conciliatory nature of this model, which seeks to offer a solu- 
tion to the tension between the quest for modernisation and a devotion to preserving valuable 
Weberian foundations of public services such as “assumptions of a positive state, a distinctive 
public service and particular legal order” (Pollitt and Bouckaert 2017:120). For NWS modernis- 
ers, the main task is thus to find a way to combine the traditional values of public administration 
with the new virtues of efficiency, responsiveness, flexibility, and adaptability. This, from a policy 
tools perspective, implies not just reaffirmation of the role of the public sector organisations as 
the driving force behind both design and delivery of public services but also the shift from rule 
following toward the holistic focus on addressing citizens’ needs (Kekez 2019). 
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The primacy of the state in the organisation-based policy tools is followed by procedural 
reliance on inputs. Management of the service delivery, hence, remains focused on the assign- 
ment of tasks to public sector organisations. With hierarchical relations with internal providers 
and rules as a central procedural and management tool, management of street-level practice 
is focused on internalisation of operating procedures. Compliance with the rules is ensured 
through practices of leadership and on-the-job training (Hill and Hupe 2014). Nevertheless, 
and, for this chapter, more importantly, in the NWS-inspired modernization of government 
sector, strict reliance on rule following in direct service provision is accompanied by shift toward 
achievement of results and toward meeting citizens’ needs (Pollitt and Bouckaert 2017; Pollitt 
2008; Kekez 2019). 

To incorporate citizens’ perspective in state-centred policy solutions, this reform model 
reaches toward the addition of collaborative and participatory elements in service delivery (Pol- 
litt and Bouckaert 2017; Pollitt 2008; Mazur and Kopycinski 2017). This, in the first place, 
implies the usage of consultations involving participation and dialogue with stakeholders, ben- 
eficiaries, and the public throughout the process of service design and delivery. Even though 
consultative efforts can operate in a largely tokenistic manner, they do have strong collaborative 
potential as they can be organised through collaborative empowering engagement with citizens 
and their associations (Johnson and Howsam 2019; Kekez, Howlett, and Ramesh 2019). In 
such cases, consultative in-house service delivery implies consensus-oriented decision-making 
and leads to actual injection of citizens’ priorities in the design and delivery of public service. 


Conclusion 


Policy reform efforts discussed in this chapter have over the last four decades simultaneously led 
to both the reduction and the enhancement of the role the state plays in modern societies. On 
the one hand, reforms introducing the production of public services and goods by external pro- 
viders have rolled back the state from the design and management of organisational policy tools. 
Creation of a space in which a variety of external for-profit and non-profit organisations can 
act as producers of public services and goods has induced the design of a broad range of indirect 
means of goods and services delivery. 

On the other hand, with the influx of organisational policy tools, thanks to which the state 
became less involved in a direct production of public services and goods, its coordination role 
has even become bigger. The need for adaptable, strategic, and effective coordination of external 
actors engaged in public service provision has brought to the fore a variety of procedural tools. 
Moreover, the same need gradually led to the recognition of cross-sectoral collaboration as a 
value-added coordination mechanism, even for the organisational tools that were inherently 
steered either through competitive contracts or hierarchy. 

With the emergence of relational contracting and a holistic version of in-house service pro- 
vision, a collaboration-based policy toolbox nowadays not only includes NPG-inspired co- 
management and co-production as the arrangements directly coordinated via collaboration, but 
it also entails tools that exist at the nexus of different governing principles. Such implementation 
hybrids hold strong potential for public value creation but, at the same time, require careful 
consideration of conditions and capacities required for their success. 


Note 


1 Review of policy reform models and aligned policy instruments in this chapter is based on the author’s 
doctoral dissertation completed at the Faculty of Political Science, University of Zagreb. 
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BIG DATA, ARTIFICIAL 
INTELLIGENCE AND THE 
FUTURE OF REGULATORY TOOLS 


Fernando de Barros Filgueiras 


The use of digital technology tools to improve governance is not exactly new. Governments have 
employed information and communication technologies for some time, and they are important 
organizational tools to retain information and use it for policy objectives and the construction 
of digital public services. Digital tools represent important organizational elements, especially in 
the policy implementation. The expansion and reach of individuals’ and companies’ connections 
on the internet, the growing capacity for data storage, and computational capacity provided the 
emergence of big data methodologies that revolutionize digital tools and their applications in 
different policy domains. 

Big data is an interdisciplinary field of knowledge involving social sciences, psychology, engi- 
neering, and computer science. The objective is to optimize the extraction of information 
from different behavioral aspects of individuals in their daily interactions on the internet or the 
information extraction in large datasets, transform them into data, and analyze them to focus on 
different purposes. Big data implies an extensive process of collecting, storing, analyzing, and 
sharing data to constitute varied applications in policies, businesses, services, and industry. The 
emergence of big data methodologies has led to applications and software resurgence based on 
artificial intelligence (AI). 

AI does not have a precise definition in the literature. AI is not the automation of repetitive 
activities carried out through codes. Nor does AI mean the reproduction of procedures fixed in 
algorithms. AI involves a process of emulating human intelligence to establish a decision or solve 
a problem. AI foundations are connected to an abstraction of the human mind to build machines 
that emulate rationality and develop cognitive functions, capacity, behavior, and thought struc- 
ture. There are different methodologies for building AI solutions, such as machine-learning 
algorithms, deep learning, facial recognition, and natural language processing. The use of these 
different methodologies implies machines’ ability to make decisions and autonomously solve 
problems. 

AI represents systems that make decisions based on learning, creativity, and increased perfor- 
mance. Applied to governments, AI represents an important tool for increasing policy outcomes. 
The application of big data and artificial intelligence in public policies finds several applications 
and has a revolutionary aspect for governments. First, big data and AI methodologies increase 
the decision-making capacity since machines can extract and analyze a large volume of data 
representing behaviors in society and provide different interventions that change the status quo. 
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Second, the application of big data and AI can change the dynamics of interaction between 
citizens and public policy, observing the performance of chatbots, autonomous vehicles, facial 
recognition systems in urban space, IoT solutions, and blockchain applications in various trans- 
actions and businesses. Third, big data and AI expand policy simulation capabilities, providing 
anticipation of demands, rules enforcement, predictability, and accuracy. 

Big data and artificial intelligence revolutionize policy instruments and provide the digital 
transformation of public policies. They optimize the discretionary and authoritative aspects of 
the tools, and they can extract and understand individuals’ behavior in society and build inter- 
ventions that alter these behaviors in the policy dynamics. Big data and AI occupy an increasing 
institutional place in policy design through autonomous decision systems. This chapter aims 
to understand the revolution of policy instruments through digital tools: understand how they 
change the dynamics of policy, connect outcomes to implementation, automate decisions, and 
imply new challenges for policy design, which is now related to the system’s design. In our argu- 
ment, big data and AI are policy instruments on the frontier of knowledge, demanding accurate 
calibration processes within the policy design. 


Regulatory Tools 


Policy tools are central to the regulation. Regulation is a complex and disputed concept (Koop 
and Lodge, 2017). Regulation comprises an expanded set of practices that aim to impose state 
authority on social and economic actors to comply with the rules. 

There are three perspectives on regulation. First, regulation comprises enacting rules accom- 
panied by mechanisms to monitor and promote compliance with these rules. Second, regulation 
is the effort of state agencies to steer the economy. Third, regulation encompasses every form 
of social control, including unintentional processes and non-state actors (Baldwin et al., 2012; 
Jordana and Levi-Faur, 2004). The regulatory governance requires policy designs that apply 
tools and policy mixes to achieve goals. Policy tools are “techniques of governance that, one way 
or another, involve the use of state authority or its conscious limitation” (Howlett, 2005, 31). 

Due to the globalization process, there is a broader trend for states to shape regulatory gov- 
ernance. As a result, states have emphasized the use of authority, rules, standards, and guidelines, 
unfolding from a previous emphasis on public ownership and direct service provision (Hood 
et al., 1999). Regulation involves a complex set of tools which aim to promote interventions 
that influence the behavior of social and economic actors. Furthermore, regulation and the use 
of regulatory tools involve complex and diverse institutional grammars, with different types of 
rules that guide the use of tools (Dunlop et al., 2021). Institutional grammar reflects types of 
rules that define positions, boundaries, choice, aggregation, information, payoff, and scope to 
define the action situations that inform policy design (Ostrom, 2005). Regulation depends on 
the creation of regulatory agencies, on delegation processes for private organizations, or on the 
creation of advisory committees. For example, environmental agencies monitor CO2 emissions, 
deforestation, air quality, and water governance rules that support interventions aimed at sus- 
tainability. Alternatively, private organizations impose industrial product standards and monitor 
the industrial process to guarantee quality for international trade. Finally, advisory committees 
establish economic parameters according to the country’s monetary and fiscal policy. In all these 
cases, regulation will occur through monitoring structures that generate information and sup- 
port the construction of policy interventions to achieve an objective. 

Tools, in general, can be framed in different ways in public policy, implying policy choices 
and alternatives (Howlett, 2019; Peters, 2018; Vedung, 1998; Hood and Margetts, 2007; Las- 
coumes and Le Gales, 2007). A helpful conception to understand regulatory tools as policy tools 
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is a cybernetic conception of governments. In cybernetic conception, information is essential 
for governments to produce knowledge, learning, and evidence to policy design. To understand 
the tools that governments apply, the analyst must circumscribe their adoption in the relation- 
ship between government and society, distinguishing the tools between detectors and effectors 
(Hood and Margetts, 2007). Detectors are all the instruments that governments use to obtain 
information inputs. After the information is processed, they produce effectors, which are the 
tools that governments use to try to have an impact on society (Hood and Margetts, 2007, 4). 

Information and knowledge are essential for the choices made by policymakers to design a 
policy that impacts society (Ostrom, 2005). The information collected by detectors enables poli- 
cymakers to make decisions, design interventions, and choose tools that aim to produce impacts 
on society, shaping the behavior of individuals and groups in society (Hood and Margetts, 2007). 
Detectors and effectors determine that policy tools are means to achieve a particular government 
objective. Policy design is a dynamic activity in which policymakers make decisions based on 
evidence, learn from the past, and relate the tools in a congruent, coherent, and consistent way 
(Capano, 2017; Howlett, 2014; Radaelli and Dunlop, 2013). Therefore, information is essential 
for the performance of the chosen tools and templates. 

The role of information in the choice and use of policy tools is essential. Digital tools detect 
information — produce inputs — and shape interventions that aim to impact society. Digital tools 
are policy tools built using digital technologies (Filgueiras and Almeida, 2021). Their use extends 
to different policy domains and involves technical and political choices. In a regulatory state, digital 
tools can be understood as mediators of the state’s authority to implement public policies. The 
emergence of big data methodology and the rediscovery of artificial intelligence (AI) increase the 
possibilities for using information for regulatory governance by automating detectors and effectors. 

The advancement and deployment of AI technologies and the possibilities brought by big 
data methodologies augment and automate regulatory governance. The expansion of these digi- 
tal technologies leads to a future in which the possibility of increasing governments’ capacities 
and automating regulatory governance is real. The future of regulatory tools requires the grow- 
ing application of AI solutions with the extensive use of big data methodologies. The purpose 
of this chapter is precisely to address this future of regulatory tools, considering the various 
dilemmas for regulatory governance. This future is not precisely regulations being formulated 
and implemented by humanoid robots but a series of dilemmas and new problems that shape the 
choice and use of digital tools in regulatory governance. 


Augmenting and Automating Regulation — Big Data 
and Artificial Intelligence 


Digital tools mean building capacities to deliver data-driven and digital-based regulation. Digi- 
tal tools promote digital-era governance (Dunleavy and Margetts, 2013) and expand policy 
outcomes through digital technologies (Veale and Brass, 2019). Amid this process of expanding 
capacities and new governance modes, two landmarks were essential. First, the big data revolu- 
tion has improved massive data collection, storage, processing, and sharing. Second, the rein- 
vention of AI technology has expanded possibilities for application in public policy. These two 
technologies have provided new directions for policy formulation, implementation, and evalu- 
ation, building new forms of communication between governments and society and, in turn, 
new forms of policy instrumentation. Emerging digital technologies have produced disruption, 
altered communication and knowledge construction, and expanded the way policy interven- 
tions constitute new templates in policy design. 
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In addition to increasing capacities, these technologies potentially automate multiple policy 
domains and interventions: in particular, regulation. Automation occurs with the design and 
deployment of autonomous systems to make policy decisions. Thus, autonomous systems are 
not just about performing repetitive tasks but also about the ability of autonomous systems to 
make decisions and define the flow of regulatory governance without any human intervention. 
For example, AI has been applied to health regulation and restaurant inspection, making deci- 
sions that impact public health (Kang et al., 2013). In addition, environmental agencies monitor 
and collect satellite imagery data on deforestation by using AI systems to analyze diverse images 
and simulate environmental control decisions. 

Gradually, AI and big data have become ubiquitous policy tools. They are developed to exer- 
cise authority. The policy automation assumption makes AI and big data the main tools of all 
regulatory tools. Moreover, AI and big data change the logic of detectors and effectors, expand- 
ing the amount of information collected and processed to define, automatically or not, policy 
design activities. Thus, AI and big data gradually become a meta-regulatory tool, comprising 
autonomous decision systems designed and employed to accomplish tasks and solve problems 
related to state authority and social control. For example, in China, the national AI strategy lists 
artificial intelligence as a tool to extend social control for broader purposes to maintain political 
order (Roberts et al., 2021; Dai, 2020). 

As meta-regulatory tools, big data and AI comprise a series of autonomous decision-making 
and tasks that frame diverse regulatory interventions. Big data and AI perform functions related 
to information dissemination in society, allocation of resources based on predictions and simu- 
lations, alteration of various organizational aspects of governments, and regulatory efforts to 
achieve compliance. As meta-regulatory tools, big data and AI are detector and effector tools, 
enabling digital technologies to frame new regulatory governance. 

The use of big data and AI as regulatory tools advances towards the definition of algorithmic 
regulation. As defined by Yeung (2018, 507), algorithmic regulation is the use of autonomous 
decision systems to regulate a domain of activities to manage risk and change behavior through 
knowledge generated by computer systems. Algorithms using big data techniques and AI sys- 
tems detect or predict violations in real-time or pre-emptively and recommend or automate 
sanctions. Thus, big data and AI are detectors and effectors for regulatory tools. Big data and 
AI aim to produce real-time or pre-emptive monitoring to detect violations and recommend 
or automate sanctions, finding applications in economic regulation, social control, and ensuring 
compliance. 


Regulatory Detectors — Big Data 


Big data is the set of methodologies and technologies for collecting, storing, processing, and 
sharing data from various sources to create an expanded information dominance and deploy- 
ment of different computational technologies (Kitchin, 2013; Mayer-Schonberger and Cukier, 
2013). In general, data collection stems from three primary sources: (1) directly, through sur- 
veillance mechanisms; (2) automatically collecting transaction data on the distributed network 
or with social media data-scraping techniques; and (3) voluntarily, through crowdsourcing 
(Kitchin, 2013). 

Data includes numbers, text, speech, and images that feed massive amounts of data to detect 
different situations in society, including the various interactions, relationships, transactions, and 
values in social networking. Two factors were essential for the big data revolution: first, the 
expansion of data storage capacity, especially with cloud computing, and second, the increase in 
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processing speed, enabling greater automatic collection capability. Big data, therefore, responds 
to societies’ datafication, in which the hyper-connection of individuals on the internet, the 
growing offer of digitized services — public and private — and the connection in social media 
enable the expansion of data sources and collection possibilities. Furthermore, big data repre- 
sents the opportunity to obtain information beyond administrative records, comprising data 
collection on digital public services transactions, social media monitoring, and data obtained 
with IoT, among others (Giest, 2017). Big data includes volume, speed, variety, exhaustiveness, 
resolution, relationality, flexibility, and scalability (Mayer-Schénberger and Cukier, 2013). 

Governments have designed data-driven policies in different policy domains. Big data meth- 
odologies have been used to formulate and design policies. As a growing trend in govern- 
ments, big data makes up the main feature of digital transformation (Giest, 2017; DeSouza and 
Jacob, 2017; Williamson, 2014). The data-driven policy requires that digital technologies col- 
lect, process, and share data (Athey, 2017). Gradually, organizations commodify information in 
distributed networks (Hess and Ostrom, 2007). Different digital technologies, such as machine- 
learning algorithms, require a vast amount of training data to produce the expected results. 
Data implies a logic of shared resources in a networked place, with a private and public nature. 
Governments use these data to develop different applications to provide services and policies 
through information control (Filgueiras and Almeida, 2021). 

Data is a resource that can be exploited to encourage different digital technologies and design 
policy from a data-driven perspective. This complex set of digital technologies comprises differ- 
ent mechanisms that collect information and transform it into data and control. Digital technol- 
ogies depend on sharing data — considered a resource — and using them to construct information 
and application by organizations (Frischmann and Selinger, 2018). Data must be understood as 
resources that enable the input of policies to deploy policy tools based on digital technologies 
(Filgueiras and Almeida, 2021). For example, the constitution of autonomous agents based on 
artificial intelligence, such as public service chatbots, image diagnostics for health care, and facial 
recognition systems for public safety, among others, require a vast amount of data to deliver pre- 
dictive decisions. Therefore, governments must mobilize massive data in all situations that allow 
autonomous agents to accomplish some policy objective. 

Big data reshapes detectors’ tools. It represents an essential digital tool to formulate differ- 
ent solutions with information and knowledge. For example, big data can solve the problem of 
understanding citizens’ political preferences when designing a policy. Big data can also be a tool 
to understand how transactions in public services take place and promote service regulation in 
governments. Moreover, big data can be a tool for co-creating policies and services, expanding 
society’s participation in indirect ways. Big data can also gather information during implemen- 
tation, enabling faster adjustment of rules and procedures, producing constant learning. When 
transforming detectors, big data is an essential meta-tool for creating other tools, quickly mobi- 
lizing data that constitutes expanded information in a granular and relational way. 


Regulatory Effectors — Artificial Intelligence 


The big data revolution has expanded data volume, granularity, and availability. It has also 
brought about artificial intelligence (AI) reinvention. Artificial intelligence does not have a 
precise concept in the specialized literature. The conceptual bases of AI are on intelligence to 
perform a task or solve a problem, which may vary according to the intelligence being conceived 
(Russell, 2019). Thus, AI is a computational technology that imitates human intelligence, but 
the way it works can vary depending on how intelligence is conceived. The assumption of AI 
development is imitation (Turing, 1950). AI is an imitation of human rationality (Simon, 1957), 


538 


The Future of Regulatory Tools 


cognitive function (Russell and Norvig, 2010), capability to perform different tasks (Minsky, 
1985), or structured thinking (Markram, 2006). AI, therefore, can mean different things. The 
concept that has been established for artificial intelligence is that of a machine’s ability to learn 
from massive data — collected, stored, and processed with big data methodologies — to predict 
and simulate different issues. Artificial intelligence means the constitution of agents — which 
perceive and act — to achieve a goal, adopting various methodologies and approaches. These 
agents perceive and build a stream of inputs converted into a stream of actions — outputs (Rus- 
sell, 2019). 

In regulatory governance, the application of AI in different domains and sectors reshapes the 
effector’s tools. AI systems play a pivotal role in the policy process to predict policy alternatives 
and solve problems. AI systems act as effectors’ tools. They predict individual behaviors and use 
the predictive ability to change expectations and introduce incentives and behavioral changes 
(Dunleavy, 2016; Kosters and Van der Heijden, 2015). AI algorithms have the power to alter 
social orders (Danaher, 2016) by exercising a technocratic authority to make or support deci- 
sions autonomously. 

Several uses of regulatory tools today comprise the application of AI. First, as a meta-policy 
tool, AI becomes ubiquitous in different solutions, producing prediction and simulation to 
solve different regulatory problems. For example, the distribution of benefits or authorizations 
can be carried out by autonomous decision systems. Second, AI can be applied to simulate the 
response to different policy issues or be used in audit systems to detect fraud and corruption and 
ensure reliability and compliance in resource use (Lima and Delen, 2020). AI can change the 
performance of authority tools by creating robust regulations. 

The advance of AI allows us to state that, nowadays, public policy solutions require artificial 
general intelligence. Artificial intelligence has a temporal and trending classification that portrays 
its progress. Artificial narrow intelligence (ANI) means that algorithms cannot learn and do not 
mold or adapt to the input data. ANI focuses on solving a single problem or performing a single 
task. Artificial general intelligence (AGI) develops the ability to learn, and algorithms adapt and 
shape solutions more flexibly, solving different problems. AGI solutions require general intelli- 
gence to achieve a variety of goals and carry out a variety of tasks. Systems are designed to oper- 
ate with adaptive learning to create other systems, solve various problems, and perform different 
tasks (Goertzel, 2014). Finally, superintelligence is possible when machines acquire awareness 
and think autonomously without the imitation requirement (Bostrom et al., 2020). 

Algorithms occupy a central place in the analysis of AI solutions in regulatory governance. 
Algorithms are sequences of instructions or steps to solve a problem or perform a task. In terms 
of artificial general intelligence, machine-learning algorithms have been developed to tackle 
complex problems through fast sorting, analysis, and pattern recognition applied to massive 
amounts of data beyond human capacity to analyze (Domingos, 2015). Machine-learning algo- 
rithms are a “field of study that gives computers the ability to learn without being explicitly 
programmed” (Samuel, 1959). Machine-learning algorithms can be supervised when a pre- 
defined dataset incorporates training examples. Machine-learning algorithms can also be unsu- 
pervised, where the algorithms build patterns and classifications related to problems without 
being trained. Machine-learning algorithms involve different designs based on deep learning, 
computer vision, rule-based systems, natural language processing, neural networks, and speech 
recognition. 

The advance of artificial intelligence places it as a meta-policy tool endowed with ubiquity 
to solve different regulatory problems and produce different impacts. The fact is that AI can 
automate policy design as it learns from the past. It has the adaptive capacity to shape solutions, 
change human behavior in society, predict and simulate new problems, and change the entire 
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logic of the application of policy tools: in particular, those related to state authority. However, 
artificial intelligence also demands design processes that cover different system logic problems, 
scope, and objectives. 


When Policy Design Meets System Design 


Public policy involves complex design activities performed by specialists and experts to create 
templates for government interventions in society. They mobilize tools to achieve policy objec- 
tives. The policy design dynamics consider the internal institutional environment and gover- 
nance modes (Peters, 2018). Design dynamics depend on the calibration of tools. Calibration 
involves patterns and rules defined in governance modes and design spaces that shape a relation- 
ship between the policy means and goals (Howlett and Capano, 2020). 

Big data and AI are emerging meta-policy tools. They also depend on design activities, 
requiring calibration and proper evaluation of the tool’s application to achieve policy objectives 
adequately. If big data and AI are the tools of the policy tools, they are also part of complex 
design activity, demanding calibration, context, time, and the observation of design precepts 
such as coherence, consistency, and congruence to produce the expected effects. The adoption 
of big data and AI in public policy makes policy design activities meet system design activities. 
This encounter requires thinking about the activities of two distinct professional fields, requiring 
a science of the artificial that “is concerned with how things ought to be, with devising artifacts 
to attain goals” (Simon, 1969, 114). 

Two styles inform the design of AI solutions. On the one hand is an empirical style per- 
formed by imitation of intelligence supported by observations of biology and psychology, 
creating complex architectures from several different modules. The scruffy is an AI design 
style embedded in confusing complexity. On the other hand, there is an analytical style, 
supported by general and organizing principles, interested in abstract conceptions of intel- 
ligence through mathematical and logical arguments to imitation. As this second approach 
is named, the neat style theorizes intelligence in ivory towers, disconnected from the real 
world (Abelson, 1981). Scruffy and neat played a profound debate on the foundations of AI 
system design, considering which approach could optimize intended outcomes. It appears 
that today, the neats have found more adherence and success, especially with complex sys- 
tems based on neural networks and building perception from data collected with big data 
instruments (Halevy et al., 2009). 

These styles for AI design consider the construction of artifacts that mobilize massive data 
to build a perception stream and produce outputs. At first, data represent the essential resource 
through which the input detection occurs so that AI algorithms produce the expected outputs, 
aiming at effects optimization. The rationale that involves the design of AI systems starts from a 
notion of utility, in which AI systems move by the rationality of cost and benefit (Russell, 2019). 
However, AI systems do not represent final or watertight solutions. Instead, decisions involv- 
ing the AI system design are taken in contexts of ambiguity, in which designers need to choose 
between different streams of problems that make AI systems solutions that look for problems 
(Filgueiras, 2021). Big data and AI applied as regulatory tools face several problems regarding the 
ambiguity and uncertainties of regulation. 


AI systems are not a neutral technology as they are politically oriented (Crawford, 2021). 
First, due to policy detectors, data are not neutral instruments but power technology (Gitelman 
and Jackson, 2013). Second, classifications that drive supervised machine-learning algorithms, 
for example, reproduce social structures of inequality. Likewise, unsupervised algorithms will 
reinforce these same inequalities by relating massive data to different questions (Joyce et al., 
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2021). The result of this process is that Al-based effectors will produce solutions or carry out 
tasks in such a way as to produce deleterious effects on society, producing ineffective policy 
solutions (Filgueiras, 2021). For example, algorithms in the United States health system have 
recommended that black patients receive less health care than white patients in the same situa- 
tion (Benjamin, 2019). 

Another example is how AI algorithms produce technological redlines, segregating society 
into spaces of exclusion (Noble, 2018). Furthermore, big data instruments exasperate surveil- 
lance systems and directly affect citizens’ privacy (Sanfilippo et al., 2020). Another extreme 
example is the emerging use of lethal autonomous weapons systems, which allow algorithms 
to make life-and-death decisions in conflict situations (Taddeo and Floridi, 2018). This set of 
problems has highlighted the need for creating governance modes that directly impact the design 
activities of big data and AI systems. AI and data governance modes guide system designers’ 
practices by changing and shaping the calibration of detectors and effectors, especially in public 
administration, where system design meets policy design (Filgueiras, 2021). 

Working with data requires governments to support and regulate the infrastructure neces- 
sary to collect, store, process, and share data. This infrastructure can be public or private. It 
makes up a shared arrangement that defines data governance. Governments may be responsible 
for collecting, storing, processing, and sharing data, mobilizing internal capabilities to handle 
data resources that support data-driven policy. Alternatively, governments can enter contracts 
with the data brokers’ industry responsible for working with data and feed government systems 
(Crain, 2018). Governments can also enter cloud computing contracts, modifying data storage 
and processing logic, impacting sovereignty (Irion, 2012). Data governance is the construction 
of rules and responsibilities for the organization and implementation of data policy. It strength- 
ens decision-making and accountability concerning data, understood as organizational resources 
(Benfeldt et al., 2020, Weber et al., 2009). Data governance will guide the calibration of detec- 
tion tools, defining organizational principles and resources to define data access rules and poli- 
cies for data qualification. 

Concerning effectors, there is a growing demand for AI governance. AI governance responds 
to ethical and regulatory challenges to guide the design and deployment of AI systems (Taei- 
hagh, 2021). The objective of AI governance is to reconcile a consequentialist logic of opti- 
mization that guides the system design — scruffy and neat — with a logic of appropriateness, 
ensuring decision-making processes, with the participation of policymakers, with transparency 
and algorithmic accountability, data governance, and validation and explicability of AI systems 
(Filgueiras, 2021). AI governance aims to guide system designers and policymakers from insti- 
tutional frameworks to design systems and link them as policy tools. This connection depends 
on the choice of policy alternatives to produce systems that make it possible to achieve goals, 
reducing ambiguities and uncertainties regarding AI in public policy. When designing AI sys- 
tems, policymakers must calibrate policy tools along with the calibration of digital tools, aiming 
for instrumentation congruence, consistency, and coherence. 

The emergence of big data and artificial intelligence as meta-policy tools has promoted the 
encounter between policy and system design. The application of digital tools requires policy- 
makers to work with opaque technologies in contexts of ambiguities. It requires a permanent 
calibration and adjustment of systems to meet those policy goals and public targets. The emer- 
gence of AI as an essential policy effector for contemporary governments opens a series of new 
challenges. They demand adequate governance modes that accurately deploy digital technologies 
and drive governments to achieve their goals. The challenges posed by emerging technolo- 
gies require institutional governance frameworks that improve the application and control of 
digital technologies during policy design processes. 
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Final Remarks — The Future of Regulatory Tools 


Big data and artificial intelligence in public policy find several challenges, but they also pro- 
vide revolutionary governments. First, big data and AI increase decision-making capacity since 
machines can extract and analyze a large volume of data by capturing varied social behaviors 
and providing different interventions that change or maintain the social order. Second, the 
application of big data and AI can change the interaction between citizens and public policy 
through the observation of the performance of chatbots, autonomous vehicles, facial recogni- 
tion systems in urban space, IoT solutions, and blockchain applications in various transactions 
and businesses. Third, big data and AI expand policy simulation capacities, providing anticipa- 
tion of demands, rules enforcement, predictability, and accuracy. Fourth, big data and artifi- 
cial intelligence revolutionize regulatory tools by acting as detectors and effectors, respectively. 
They optimize the discretionary and authoritative aspects of the tools, and they can extract and 
understand situations in society and build interventions that alter target population behavior in 
the policy dynamics. Finally, big data and AI occupy an increasing institutional place in policy 
design through autonomous decision systems. Digital tools have revolutionized regulation as 
they change policy dynamics, connect outcomes to implementation, automate decisions, and 
pose new challenges for policy and system design. In our argument, big data and AI are meta- 
regulatory tools on the frontier of knowledge, demanding accurate calibration processes within 
the policy design. 

As technological development advances in big data and artificial intelligence, new chal- 
lenges arise when governments employ digital tools. In the current scenario, the advance of 
artificial general intelligence poses a series of challenges for policymakers. The main one is 
understanding how to calibrate and govern an artificial general intelligence that works with 
dynamic algorithms, which change as the data presents new trends and problems. AGI is 
endowed with full autonomy to make decisions and carry out tasks. If AGI is autonomous, 
plastic, and flexible, how to calibrate these digital tools that gradually take on the contours of 
agency? AI governance must target several ethical and political issues by creating institutional 
frameworks shaping actors’ behavior in designing and deploying AI applied to public policy. A 
future dominated by superintelligence creates new problems with conscious and fully autono- 
mous AI, which demands a transition to another type of society and different policy vectors 
(Bostrom et al., 2020). 

The future of regulatory tools is that big data and AI algorithms become institutions. Algo- 
rithms gradually occupy an institutional space in societies, changing different aspects of social 
life, shaping collective and individual human behaviors. As institutions, algorithms work as auto- 
matic (often autonomous) decision systems that define what is allowed, hindered, facilitated, or 
made impossible. Also, they define positions within society’s organizational structures. More- 
over, algorithms gradually become the norm — formal or informal — and they require compli- 
ance by actors. In other words, the big data and AI algorithms will be regulation tools and the 
regulator itself, establishing a new power agency in which compliance and social control depend 
on algorithms that codify rules and act to enforce them. 
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THE FUTURE OF FINANCIAL 
TOOLS 


Will Social Impact Bonds Yield Results? 


Daniel Bromberg 


Social impact bonds (SIB) emerged in the early 2000s in the United Kingdom. Similar to a 
pay-for-performance scheme, SIBs establish a financial structure in which the service provider 
receives payment — and potentially performance bonuses — upon the achievement of targeted 
goals. They have primarily been utilized to attain long-term social goals which have not gained 
support through traditional political processes. Advocates of SIB argue that they can address 
pressing social needs through a collaborative model that spreads risk across both the public and 
private sectors. Ideally, they are used to offset government expenditures while simultaneously 
stimulating growth. The combination of these factors is what many think makes these an inno- 
vative tool. Detractors suggest they are nothing more than a performance-based contract and 
rarely alleviate government risk. This chapter will provide an overview of the historical use of 
SIBs, their growth in use throughout the world and the prospects of SIBs playing a role as a 
social policy tool. 


Introduction and Definition 


Social impact bonds (SIB) emerged in the early years following the great recession in 2008. As 
Neyland (2018) notes, times of “crisis and austerity became a means to give new momentum to 
a series of experimental ways to shape the social investment market” (p. 494). The goal of SIBs 
is to offer an alternative funding mechanism to address challenging social problems. 

Definitions of SIBs abound, often clouding the nuts and bolts of the instrument. The OECD 
offers the following definition: 


A SIB is an innovative financing mechanism in which governments or commission- 
ers enter into agreements with social service providers, such as social enterprises or 
non-profit organisations, and investors to pay for the delivery of pre-defined social 
outcomes (Social Finance, 2011; OECD, 2015). More precisely, a bond-issuing organ- 
isation raises funds from private-sector investors, charities or foundations. These funds 
are distributed to service providers to cover their operating costs. If the measurable 
outcomes agreed upfront are achieved, the government or the commissioner proceeds 
with payments to the bond-issuing organisation or the investors. In reality, the term 
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“bond” is more of a misnomer. In financial terms, SIBs are not real bonds but rather 
future contracts on social outcomes. 
(OECD, 2016, p. 4) 


The United States Department of Labor defines them as follows: 


The Pay for Success (PFS) model is a new way of financing social services to help 
governments target limited dollars to achieve a positive, measurable outcome. Under 
the Pay for Success model, a government agency commits funds to pay for a specific 
outcome that is achieved within a given timeframe. The financial capital to cover 
the operating costs of achieving the outcome is provided by independent investors. 
In return for accepting the risks of funding the project, the investors may expect a 
return on their investment if the project is successful; however, payment of the com- 
mitted funds by the government agency is contingent on the validated achievement 
of results. 


While the terms bond and innovative may imply certain conditions, an SIB is neither a bond 
nor necessarily innovative. An SIB is a pay-for-performance contract in which an investor, typi- 
cally coupled with government and philanthropic funding, provide capital to fund a specific 
project. If specific performance targets are reached, the funder receives their initial investment 
and a return on their investment. The language attached to SIBs has caused both confusion and 
criticism. The confusion stems from the implication that this is a market-based financial tool. 
The criticism stems from those who claim SIBs further promote the financialization of govern- 
ment. Proponents of SIBs note that the “innovation” is what allows governments to leverage 
otherwise inaccessible capital to address pressing social problems. 

The first SIB was launched in 2010 in the United Kingdom by Social Finance UK. The ini- 
tial SIB focused on reducing offender recidivism at HMP Peterborough. Similar initiatives have 
emerged in the United States, termed Pay for Success (PFS), and in Australia, termed Social 
Benefit Bonds (Dear et al., 2016). The first US-based SIB was issued by the city of New York 
in 2013 to reduce recidivism at Rikers Island prison by young (16-to-18-year-old) offenders. 


Tool Description 


An SIB is a policy tool structured to bring several actors together. As Lester Salamon (2002) 
defines, a “tool, or instrument, of public action can be defined as an identifiable method through 
which collective action is structured to address a public problem” (p. 1642). An SIB most closely 
resembles a pay-for-performance (PFP) contract in which the service provider receives pay- 
ments based on certain pre-established targets. The distinction between an SIB and a PFP is that 
an SIB is reliant on multiple actors to finance, evaluate, and provide services to the target client. 
The first step is for the government entity to contract with an intermediary who will help facili- 
tate the relationship. Following this initial contract, the intermediary will initiate the remainder 
of the process as diagramed in Figure 42.1. 

The key actors are a government, a non-profit service provider, investors, an external evalu- 
ator, and the facilitating intermediary. The process is quite simple. A service provider and an 
intervention are identified, capital is raised to fund the service provision and targets are identi- 
fied for the service to reach. In theory, the targets identified will save the government money, 
creating excess capital. The excess capital is then utilized to return the initial investment with 
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1 An intermediary issues the SIB and raises capital from private investors. 
2 The intermediary transfers the SIB proceeds to nonprofit service providers, 


which use the funds as working capital to scale evidence-based prevention 
programs. Throughout the life of the instrument, the intermediary would 
coordinate all SIB parties, provide operating oversight, direct cash flows, 
and monitor the investment. 


3 By providing effective prevention programs, the nonprofits improve 
social outcomes and reduce demand for more expensive safety-net services. 


4 An independent evaluator determines whether the target outcomes have 
been achieved according to the terms of the government contract. If they have, 
the government pays the intermediary a percentage of its savings and retains 
the rest. If outcomes have not been achieved, the government owes nothing. 


5 If the outcomes have been achieved, investors would be repaid their principal 
and a rate of return. Returns may be structured on a sliding scale: the better 
the outcomes, the higher the return (up to an agreed cap). 


Figure 42.1 Social Impact Bond Processes 


Source: A New Tool for Scaling Impact: How Social Impact Bonds Can Mobilize Private Capital to 
Advance Social Good, Social Finance 


a predetermined return based on various target thresholds. Promoters of SIBs note that by 
leveraging private capital, the financial risk is shifted from service providers and governments, 
creating an otherwise unattainable solution. As Jeffrey Liebman puts it, 


Today, most providers would be hard-pressed to come up with sufficient capital to pro- 
vide services up front and only receive payments after performance targets were met. 
And most social service providers would be unable to absorb the risk of failing to meet 
performance targets. But in a social impact bond scheme, private investors provide the 
upfront capital and absorb most of the risk. 


(p. 2) 
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Size and Scope of SIBS Worldwide 


The most detailed tracking of SIBs is done by Social Finance UK — the initiating organization 
for the first SIB developed. The Brookings Institution also tracks SIBs but offers a broader 
view than Social Finance UK. The organizations offer different schemes to classify SIBs; there- 
fore, both are used to provide a comprehensive understanding of the current conditions. Social 
Finance UK is tracking 138 SIBs in more than 25 different countries. Most of these SIBs are 
in the UK (47) and the US (26), with the Netherlands (11) and Australia (10) the only other 
countries with a double-digit number of SIBs. All the other countries have fewer than 10, with 
most having 5 or fewer SIBs. According to the same Social Finance UK database, SIBs have most 
dominantly been used in workforce development (44). The next highest categories are housing/ 
homelessness (23), health (22), and child and family welfare (20). Education, criminal justice, 
poverty, and environment round out the remainder of SIBs issued. 

The Brookings Institution has been tracking and evaluating the progress of SIBs since 2010 
and currently identifies 201 SIBs. Leading this project, Gustafsson- Wright (2020) reports there 
has been $421 million in initial investments totally, over $460 million committed in return on 
investment. The average investment is about $3 million, which is skewed slightly due to a hand- 
ful of very large impact bonds (over $20 million). There was also steady growth in impact bonds 
until 2017 and 2018, when the introduction of new bonds doubled to 45 and 48, respectively 
(Gustafsson-Wright, 2020). Target populations of SIBs tend to be those grouped in younger age 
brackets with over 100 bonds reaching populations under the age of 30 (Gustafsson-Wright and 
Osborne, 2020). Further Gustafsson-Wright and Osborne (2020) note that over two million 
people have received services from SIBs with populations nearly split between high-income 
countries and low-income countries. 


Example Cases 


HMP Peterborough 


In 2010, HMP Peterborough became the first SIB launched worldwide. The focus of the SIB 
was on a sample of 1,000 offenders sentenced to 12 months or less in Peterborough Prison (Social 
Finance UK). The individuals were provided with a host of services both prior to release and after 
release. Social Finance UK explains, “Working with offenders both pre- and post-release, it provides 
a holistic service including support with housing, training and employment, parenting, substance 
abuse, and mental health problems.” The initiative had a minimum threshold of a 7.5% reduction in 
rates of recidivism. This was calculated based on a matched sample — matching those receiving the 
services at Peterborough with a like sample of those not receiving the services at a different prison.' 
This was evaluated by independent auditors on two waves of individuals. According to Anders and 
Dorsett (2017), the combined reduction in recidivism for the group who received the services was 
9.02% — higher than the 7.5% reduction threshold —triggering a return on investment. All inves- 
tors received their initial principal and a 3% return. By most accounts, HMP Peterborough can be 
seen as a success. Service recipients recidivated at a lesser rate than those offered other interventions. 
Some criticize the evaluation model, but most acknowledge the success of the effort. 


Rikers Island 


The first SIB launched in the US was at Rikers Island — a prison in New York City. Both 
the arrangement and the outcome are what detractors frequently cite about the problem with 
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SIBs. Like other SIBs, there was a government entity, private investors, a service provider, and 
an independent evaluator. However, in addition to the private investor, there was also a loan 
guarantee provided by a private philanthropy, reducing the risk to the investor by 80%. The 
intervention, unlike the Peterborough example, was cognitive behavioral therapy provided to a 
group of offenders. Ultimately, the Rikers Island SIB failed to reach initial thresholds of success, 
causing the termination of the program. According to Vera, the independent evaluator, 


The program did not lead to reductions in recidivism for participants. The change in 
recidivism for the eligible 16- to 18-year-olds, adjusted for external factors, was not statis- 
tically significant when compared to the matched historical comparison group. Further- 
more, the 19-year-olds and the study group (16- to 18-year-olds) displayed similar trends 
in rates of recidivism over time, indicating that any shifts were the result of factors other 
than the ABLE program. The program did not reduce recidivism and therefore did not 
meet the pre-defined threshold of success of a 10 percent reduction in recidivism bed days. 

(Vera Institute of Justice, 2015, p. 2) 


One can interpret the failure of this program as evidence that such initiatives are fraught with 
challenges. MDRC, the intermediary that worked on the project, noted that the challenges 
related to the intervention became evident early and offer some lessons for all to learn about 
SIBs and the Rikers SIB specifically (Berlin, 2016). They suggest that there might have been 
a cultural shift at Rikers which may leave a legacy. There are other lessons that organizations 
involved have gleaned from the experience. John Anderson, who leads Bloomberg Philanthro- 
pies Government Innovation Programs, one of the project investors, writes that the Rikers SIB 
created an opportunity to try an innovative approach to a persistent public problem without 
taxpayer dollars and “even when budgets are tight.” He further notes that due to the financial 
support, the organizations were able to collect key data from which to learn — a luxury that is 
often missing from government-run programs. 


The Commonwealth of Massachusetts 


The Commonwealth of Massachusetts has initiated multiple social impact bonds, with the largest 
being a $22-million project to reduce recidivism. In addition, the Commonwealth has a workforce 
development project initiated for about $12 million and a project focused on housing and homeless- 
ness for $3.4 million. While the structure of each of these projects is different, each combines capital 
from for-profit investors alongside philanthropic contributions. In the case of the Massachusetts 
recidivism project, there are three investors — Goldman Sachs ($9 million — for-profit), the Kresge 
Foundation ($1.5 million — non-profit), and Living Cities ($1.5 million — non-profit). The structure 
of the agreement is in the form of a loan for which Goldman Sachs is the “senior lender” and there- 
fore recoups their loss prior to the other lenders. In addition to the private investment, there is addi- 
tional philanthropic support totaling $6 million. Lastly, this project is reliant on a US Department of 
Labor grant of $11.7 million. The project was supposed to end on December 31, 2019, three years 
after it was initiated; however, a statement by the intermediary notes that they extended to project 
for five more years — through 2024 — due to slow enrollment in the initial phase of the project. 


Other Projects 


Within the US, many states have enacted legislation to enable these arrangements to take 
place. Further, there have been increased funding and incentives provided by the US federal 
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government since 2014. According to Results for America, SIBs offer bipartisan appeal, which 
has led to the creation of three pieces of legislation: the Second Chance Act, the Workforce 
Innovation and Opportunity Act, and the Every Student Succeeds Act. Since 2014, these pieces 
of legislation have enabled the Corporation for National and Community Service, the Depart- 
ment of Labor, and the Department of Education to promote and fund SIBs across the US. In 
2018, the US Congress passed the Social Impact Partnerships to Pay for Results Act (SIPPRA). 
As part of the act, Congress appropriated $100 million to fund social impact partnership dem- 
onstration projects. 


Literature 


While many ideas have emerged about SIBs, Fraser et al. (2018) identify three major themes. In 
an extensive review of both the academic and gray literature, they examined over 80 articles on 
SIBs. First, there are those that discuss the “competing public and private values.” This is often 
framed in similar terms as discussions about New Public Management (NPM) and the utiliza- 
tion of businesslike techniques in government. The second emergent theme relates to results- 
based management and the reliance on performance metrics to manage. As there is an extensive 
literature on performance management in public administration, there is much to say about its 
reintroduction utilizing a novel tool. Lastly, Fraser et al. (2018) identify articles written about 
risk shifting. That is, SIBs offer an opportunity to shift the financial risk from the public to the 
private sector. As with any new initiative, there remains a good deal of variation, allowing for 
each of these themes to emerge, dependent on the particular way the tool is used. 

SIBs have been adaptable to various environmental constraints, be they political or financial 
in nature. Therefore, they offer a degree of “strategic ambiguity” (Tan et al., 2021, p. 2), allow- 
ing them to shape shift dependent on needs. For one audience, they may be the financialization 
of the public sector, whereas for another, they may be a philanthropic effort aimed at solving a 
social ill (Tan et al., 2021). This has led others to call SIBs an “imaginative financial innovation” 
(Sinclair et al., 2021, p. 11), implying a sort of fantastical association with the tool. Regardless of 
the description, the result leaves people confused as to the nature of the tool itself. 

By using the term bond, there is an immediate association with a financial instrument, but an 
SIB is, at best, a third cousin to an actual bond. As Fraser et al. (2018) write 


SIBs do not follow the financial logic of a “bond,” as the return is contingent, rather 
like equity, on the outturn of the projects being financed. And there is yet no indica- 
tion that any significant financial market for SIBs may develop between financiers, after 
the project is initially established. 


(p. 5) 


The linguistic gymnastics that one must do to describe an SIB as a financial instrument sug- 
gests the inconsistency between the name and the mechanism. Maier and Meyer (2017) note that 
there are many variations in the how an SIB is structured. They write, the “investor may be 
philanthropic, profit-, or blended value-oriented. The investor may take on all or a part of the 
risk of non-performance, with or without guarantee of principal, with longer or shorter time 
to maturity” (p. 1). They continue to suggest that SIBs have characteristics “like a derivative,” in 
that the value is only seen once a “particular event” takes place (p. 1). Then again, maybe it is 
not like a derivative because there is “up-front capital that covers” the cost of the project (Maier 
and Meyer, 2017, p. 1). Bromberg and Justice (2016) suggest that a close comparison might be a 
pension obligation bond (POB), in which bonds are issued to cover underfunded pension funds. 
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While this is not the optimal solution as it essentially indicates a lack of upfront funding, it does 
solve an immediate problem. The implication is that if these entrenched social problems were 
addressed initially, creating an innovative funding mechanism would not be necessary. Bromberg 
and Justice (2016) also note that POBs “disguise risks, conceal and inflate costs and commit 
future legislators and taxpayers to bear the burden of their predecessors’ deferral of timely and 
adequate investment” (p. 195). The primary difference is that the risk in POBs has to do with 
the financial stability of pension plans, whereas the risk with SIBs is very squarely on the indi- 
viduals receiving an experimental treatment. 

By integrating “philanthropy, venture capitalism, performance management, and social pro- 
gram finance into an innovative new mix” (Warner, 2013, p. 304), SIBs do introduce a new 
mechanism to fund social programs. The mechanism, however, carries with it considerable 
concerns. There exist some of the typical criticisms levied at businesslike practices used in the 
public sector. Tan et al. (2021) note the difference in openness between financial instruments 
and traditional government policy tools. Whereas financial tools are built on closed exchanges, 
public sector tools are transparent. This creates scenarios in which accountability may be at least 
variable but, at its worst, untenable. There are also several considerations related to relying on 
specific measurement standards and targets. As Warner (2013) reminds us, there are issues related 
to skimming populations to ensure targets can be reached. There are also attempts to game the 
system by cherry-picking specific individuals (Carter, 2021). Moreover this can lead to increased 
“transaction costs of program design, budget liability and risk, and potential stifling of further 
program innovation in order to ensure continued private returns” (Warner, 2013, p. 304). 

The transaction cost of an SIB is something one must carefully consider. At a minimum, 
there are four partners (government, investor, evaluator, service provider) involved in an SIB. 
More than likely, there are twice as many partners. As Demel (2012) writes, “Think about the 
resources expended by each of these parties, and their lawyers, in the negotiation and execution 
of a SIB” (p. 505). This cost is often increased in the initial phases of the SIB, both for govern- 
ments and for all other parties (Giacomantonio, 2017). These increased costs have the potential 
to drive investors toward other instruments like basic grants, which will more than likely have 
lower transaction costs (Giacomantonio, 2017). 

The Massachusetts recidivism SIB offers a telling example. First, there are multiple inves- 
tors, multiple non-profit organizations, two evaluation organizations, and multiple government 
entities. The Commonwealth initially received a grant from the federal government that carries 
with it several initial and ongoing transaction costs. Then, once that was initiated, a few other 
agreements and contracts had to be established. In the first year alone, Pandey et al. (2018) con- 
clude that transaction costs in the Massachusetts recidivism SIB reached 30% of project costs. 
Giacomantonio (2017) suggests most grants carry a transaction cost of 12%. What incentive 
exists for impact investors to use a tool that drives up transaction costs? One of the only viable 
answers is that governments absorb those costs, relieving private investors. Therefore Giacoman- 
tonio (2017) concludes, 


Based on the analysis in this paper, under the kinds of conditions in which one should 
reasonably imagine a government choosing to use a SIB, it is unlikely that SIBs could 
ever represent a sound investment strategy for an investor interested in generating even 
a modest return, unless substantial incentives or guarantees are provided by govern- 
ment or third parties (in which case, the financial viability of these vehicles has to be 
questioned, particularly to governments such as the UK’s that are operating under 
austerity). 


(p. 64) 
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Unlike traditional bonds, as political risks increase, governments are less likely to offer large 
enough incentives to encourage proper investment. This leads to multiple reactions by both 
markets and governments. Rangan and Chase (2015) note two effects. First, governments may 
recognize high performance and high rates of return, encouraging them to look for more effi- 
ciencies. In doing so, they may heighten targets and or reduce rates of return. This would lead to 
decreased investment as the upside for investors would be limited. Hence, the natural tendency 
of governments to reduce risk may still drive behavior related to SIBs. Beyond these, there are a 
host of more generalized arguments like those levied at NPM initiatives. In addition, a new set 
of concerns arises specifically related to SIBs and the social problems they are trying to alleviate. 

Mainly, there are ethical considerations which concern many. Morley (2021) suggests that the 
tool itself may not be unethical 


but that many SIBs are nevertheless at high risk of being unethical because they are 
characterized by information asymmetries, power imbalances and financial incentive 
structures, any of which can lead to a failure of informed consent, poor decision-mak- 
ing by vulnerable individuals, unfair contractual arrangements or a denial of service. 
(Morley, 2021, p. 14) 


Ultimately, SIBs are experimenting on vulnerable populations. They incentivize testing tech- 
niques by leveraging private investment and leaving individuals at risk. Further, as Sinclair et 
al. (2021) write, these tools are transforming “service users into ‘fictitious commodities” (p. 3). 
While the failure of a service might mean a decreased return for an investor, the failure of that 
service might mean returning to prison for the client. 

The logic that hinges success on the individual presents additional ethical considerations. 
The basic argument is that by shifting responsibility to these individuals, SIB solutions fail to 
address the root of the problem (Dowling, 2017; Ogman, 2019). Take, for example, the Rik- 
ers Island SIB. Behavior modification therapy was implemented “without having first shown 
their [offenders’] behaviors to be the cause of their incarceration in the first place” (Ogman, 
2019, p. 12). Clearly, root causes of behavior have both individual and societal contexts. One 
may question the ethical wisdom of decontextualizing these individuals from society. This logic 
further destroys what many believe to be the role of governments — to build supports rather than 
to solely “reduce welfare dependency and foster the self-sufficiency of individuals” (Dowling, 
2017, p. 17). The creation of targets aimed at individual behaviors furthers the idea that the solu- 
tion can be “found at the level of its symptom” (Dowling, 2017, p. 22), rather than more broadly 
addressing the systematic problems. Ultimately, while many may see this as a reasonable shift 
toward individual responsibility, it is coupled with a more concerning broader shift in capital. 

Both government and philanthropic capital have been used to secure private investment. In 
certain cases, such capital has provided downside protection for investors along with increased 
profit. Ogman (2019) suggests that this model does not create “shared value” for both public 
and private actors. Rather, by creating loan guarantees and paying for the high transaction costs, 
private investment takes on little risk. Hence, governments and non-profits are taking limited 
capital and securing the risk of private investment. They are furthering “(a)n entrepreneurial 
theory of social change” that “puts centre-stage private individuals as agents of social change. 
These private individuals are often high-net worth individuals who shape public agendas based 
on their views of how society should be organised, using their private wealth to do so” (Dowl- 
ing, 2017, p. 23). Ultimately, this view suggests that not only do government and philanthropic 
investment continue to pay for social services but, in doing so, they relinquish their decision- 
making authority to choose the intervention. 


553 


Daniel Bromberg 


SIBs are not without their supporters, and those supporters make many claims. The initial 
claim is that governments are unable to make appropriate investments to track performance 
and make decisions based on results, allowing inadequate programs to sustain over long peri- 
ods of time (Liebman, 2011). SIBs, on the other hand, place evidence-based decision-making 
at the center, allowing outcome achievement to drive decision-making. Further, SIBs shift 
risk to the private sector while increasing funding to solve intractable social problems. Lieb- 
man (2011) identifies six challenges that governments traditionally face considering social 
problems related to timing and measurement of programs. SIBs, according to Liebma, have 
the potential to alleviate if not remove all these challenges. By raising private capital and using 
professional evaluators, SIBs increase government and non-profit service providers’ willing- 
ness to take risks that they would have otherwise been unable to take. Both Liebman and 
others offer criteria as to when a SIB might be the best approach. Social Finance UK offers 
the following criteria: 


Identifiable populations with complex, cross-agency needs, who require tailored inter- 
ventions, are not being served. . . . 

Current spending has poor or undetermined outcomes. . . . 

There are high financial and political costs to society and government in not addressing 
the social issues. . . . 

There is a benefit to using external investment to provide risk capital and assume innova- 
tion and implementation risk for new or evidenced-based programs. . . . 

There are social sector partners who can deliver effective services, but there is an element 
of uncertainty about what outcomes can actually be achieved. 

(Dear et al., 2016, p. 13) 


Similar reasoning is offered by Results for America. In promoting SIBs, they suggest that 
these tools are not only a viable solution for social ills but also gain bipartisan support. RFA 
argues that they offer effective strategies that reduce government waste. There is some evidence 
that SIBs are finding success. Gustafsson-Wright et al. (2020) reports that out of 49 completed 
SIBs, 33 have either reached outcomes to a point at which the original investment has been paid 
or, in certain cases, a return has been provided. This indicates that, at least based on the initial 
design, SIBs are doing what the contract sets out for them to do. 

Gustafsson-Wright et al. (2020) do caution that this is not necessarily evidence that the social 
problems are being solved. They note that even if SIBs reach their intended outcomes, there are 
limitations to what we might be able to say. After an extensive review of the literature, Fraser 
et al. (2018) write, 


To date, across all active SIBs, there has been very little rigorous counterfactual com- 
parison of SIBs versus alternative methods of finance to deliver the same service to the 
same type of users, and thus a lack of evidence of costs and benefits compared with the 
alternative approach to procurement. 


(p. 16) 


As Gustafsson-Wright et al. (2020) write, 


[I]t can be said that outcomes were achieved (or not) in these impact bonds, but not 
that this was the result of the impact bond per se. It is not even possible to attribute the 
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majority of the outcomes achieved to the social intervention, as very few projects have 
conducted rigorous impact evaluations of the interventions. 


(p. 20) 


Future of the Tool 


The attractiveness of SIBs grows in times of austerity. As the world continues to battle a global 
pandemic less than a decade after emerging from one of the largest recessions in history, it would 
be foolish to dismiss the challenging times that lie ahead. As an OECD working paper notes, “In 
times of budgetary constraints combined with aggravating social challenges, SIBs have emerged 
as an innovative financing mechanism representing a complementary approach to traditional 
social policy interventions, notably in areas where there are government and market failures” 
(OECD, 2016, p. 22). The tool seems to be here to stay, at least in the short term. Therefore, it 
is incumbent upon both the practitioner and academic community to identify potential oppor- 
tunities and provide sound guidance. 


Research and Evaluation 


There exists a contradiction in the academic literature as opposed to the narrative from the 
field. For example, Tan et al. (2021) write, “While SIBs expand, there is limited evidence that 
they produce better outcomes for service recipients, or are more cost-effective than direct 
public financing for public services” (p. 5). Fox and Morris (2021) note several considerations 
related to the evidence that SIBs produce results. First there is great variation in the type of 
evaluation that is done, which, most often, is determined by the evaluator and the funder. 
Therefore, in the US, the randomized control trial (RCT) is frequently used, whereas in the 
UK, the primary mechanism has been matched data sets. These data sets are typically con- 
ducted by using historical administrative data, which may not have been rigorously collected 
or collected with the purpose of comparison. Therefore, attribution of the intervention is 
impossible to determine. While an RCT may help with attribution, they are limited in what 
else they might be able to explain. 

The paradox, however, is that SIBs are dependent on program evaluation. As Gustafsson- 
Wright et al. (2020) note, “Stakeholders in early social impact bond (SIB) projects cited the 
opportunity to promote a new culture of monitoring and evaluation in social service deliv- 
ery, including improved data collection and management, as a reason for championing the 
model” (p. 7). Moreover, stakeholders report that this has bred a data-driven culture, and SIBs 
are responsible for this growth (Gustafsson-Wright et al., 2020). This creates the potential to 
coalesce around an idea. As Fox and Morris conclude (2021), there is a good deal of data related 
to effective evidence-based interactions. These are infrequently consulted in design and evalu- 
ation. Further, they note that the administrative data sets, often consulted for matched set data, 
are imperfect and were not intended for their new purpose. This might be convenient and 
cheap, but it is ineffective at giving us results. 

The lack of rigorous data identified by scholars and the seeming desire for rigorous data 
identified by practitioners suggest a future in which developing a rigorous data-driven culture 
is possible for social interventions. SIBs have been good at attempting to use outcome data 
rather than basic outputs in their evaluation. SIBs also have the advantage of extending contracts 
beyond short political time frames. The presents a scenario in which real gains can be made 
and documented. Evaluations should use RCT alongside other methodologies to capture the 
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attribution of the intervention coupled with more contextual evidence, allowing both scholars 
and practitioners to learn how best to support our most vulnerable populations. 


Emerging Economies 


One of the criticisms levied at SIBs is that they divert both philanthropic and government 
resources away from traditional financing of social service programs. While the evidence of this 
is not clear, there are many who suggest that SIBs create an avenue for impact investing that 
otherwise would not have been available. The principal behind these activities is more about 
“impact-seeking rather than return seeking capital” (Rangan and Chase, 2015, p. 29). If there 
are returns, they are reinvested into the service activities and the interventions. To date, the bulk 
of the capital investment has been driven to organizations with capacity, but there are indica- 
tions that shifts away from this may be emerging. Gustafsson-Wright et al. (2020) write about a 
health impact bond in India aimed at building capacity in an organization to comply with India’s 
accountability requirements. Further, Munoz and Kimmitt (2019) write about the emergence of 
SIBs in Chile and, more broadly, Latin America. While there are a broad range of issues related 
to SIBs in Latin America, more central to their argument is the how SIBs are encouraging 
growth of social entrepreneurs engaging in this problem solving. In many cases, governments 
are not central to solving these challenges. 

Impact bonds are emerging, both with and without government. Development impact bonds 
(DIBs) offer an alternative approach in the developing world. Rather than relying on govern- 
ment to make payments, both philanthropic and non-governmental organizations are establish- 
ing DIBs. Thus far, according to the Brookings Institution, DIBs represent a small share of the 
impact bond market, but they offer a potential growth area. In certain developing countries, 
these alternative models are preferred to circumvent or support the lack of capacity from gov- 
ernments. That said, when governments are involved, enabling structures need to be created. 
That might be through the establishment of “outcome funds” or funds dedicated to PFP con- 
tracts or, more directly, by establishing a regulatory framework in which SIBs can flourish. The 
US passed the SIPPRA in 2018, not only establishing the regulatory framework but also appro- 
priating $100 million to fund new initiatives. 


Final Cautions 


If the SIPPRA is any indication of how more advanced economies will approach SIBs, then we 
can expect continued growth of the tool. While this may spur innovation, it is bound to spur 
the growth of a new type of organization — the intermediary. Like the capacity needed to retain 
large government grants, SIBs are reliant on complex sets of partnerships that non-profits are ill 
equipped to establish. By design, SIBs require an intermediary to manage these relationships. 
There are currently a handful of organizations that dominate this market. New competition may 
emerge, but we will primarily see these established organizations continue to capture most of 
the funding, hence playing an outsize role in determining which projects are funded and which 
remain unfunded. 

SIBs present an opportunity to steer funding toward service providers that have demonstrated 
successful interventions, but they do so at the risk discharging governments of their responsi- 
bilities. It is important to remember that these wicked problems that SIBs offer the potential 
to solve are frequently due to market failures. Masking their solutions in financial language is a 
red herring. The only transferred risk is political, in that politicians can shift political risk to the 
funder, providing cover should the intervention fail. SIBs are not a financial instrument; they 
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are a political smokescreen. They cater to the uncomfortable reality that many will only offer 
to feed a hungry person if that person’s productivity exceeds the cost of the food. It sets a dan- 
gerous precedent in which evidence-based practice is conflated with financial risk assessment. 
Evidence-based decisions related to social interventions should be dictated by experts, such as 
trained public managers, social workers, or psychologists. If governments and funders are serious 
about employing evidence-based practice, they would be better off diversifying their portfolios 
and recognizing that not all evidence points to a financial return on an investment. 


Note 


1 See J. D. Anders and R. Dorsett, R., “HMP Peterborough Social Impact Bond-Cohort 2 and Final 
Cohort Impact Evaluation,” The Commonwealth of Massachusetts, 2017, for a detailed explanation of the 
matching. 
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THE FUTURE OF 
INFORMATIONAL TOOLS 
WITH BIG DATA INFORMATICS 


Opportunities and Challenges 
for Evidence-Based Policymaking 


M. Jae Moon and Sabinne Lee 


For the past decade, big data analytics and powerful informatics have offered compelling oppor- 
tunities for evidenced-based public policy, primarily thanks to the unprecedented availability 
of policy-related big data, which is both quantitatively and qualitatively different from conven- 
tional data in terms of its scope, scale, structure, and impact. With growing computing power 
and analytical tools, big data have become increasingly important original materials that are 
utilized, examined, and interpreted for identifying current and future policy problems, analyz- 
ing the positions of stakeholders, assessing policy tools and alternative solutions, and evaluating 
policy effectiveness. As a powerful policy tool, the availability and utility of big data offer great 
potential for improving not only the quality of policy choices but also policy outcomes, creating 
an optimistic perspective on big data-based new policy sciences. However, some still remain 
reserved and somewhat skeptical because of continued political gaming in policy decisions and 
implementation processes, as well as limitations of informatics and biases often found in big 
data. This chapter reviews the prospect of big data—based new policy sciences as evidence-based 
policymaking and then examines their opportunities and challenges based on selected cases. 


Introduction 


Information as a policy tool has been revolutionarily changing and expanding in terms of its 
availability, scalability, and applicability of information and data. In particular, the technologi- 
cal advancement of informatics and artificial intelligence has opened new doors to the future 
of information tools. Once considered more subtle, indirect, and lenient than other policy 
tools, information tools have become more powerful, direct, and specific as policymakers pay 
increasing attention to the power of data analytics and the significance of evidence-based policy, 
particularly in the information age. 

Since 2015, for example, the Department of Culture, Sports, and Tourism of Korea and 
Jeonju City, as well as Jeju Island, have developed a customized tourism policy by analyzing 
tourists’ patterns of attraction sites using big mobility data. For this analysis and prediction, big 
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data were collected from various sources. For instance, mobile population data were collected 
from telecommunications companies, and the sales data of card companies were used to analyze 
the characteristics of tourists’ sales patterns. In addition, Jeonju City collected data from social 
network services, such as Twitter and weblogs, and collected weather forecasts and oil prices to 
revitalize local tourism from various perspectives. Hanok Village was revealed to be crowded 
with tourists, especially in May and October when many festivals were held. Analysis dem- 
onstrated that transportation was needed to connect South, North, East, and West Jeonju to 
disperse tourists. As a result, Jeonju City announced that tourists increased by more than 50% in 
one year. In sum, through big data analysis, Jeonju City was able to predict tourists’ demands and 
establish a relevant public policy that attracted them to visit Jeonju City based on these predic- 
tions. The success of this analysis is because big data technology enables fragments of related yet 
heterogeneous information to be matched and linked to each other quickly (Hochtl, Parycek, & 
Schollhammer, 2016). 

However, despite the potential of big data in public policy decision-making and prediction, 
big data informatics does not receive much attention from policy decision-makers and policy 
scientists. For instance, Gupta, Kar, Baabdullah, and Al-Khowaiter (2018) reported in their 
literature review analysis that, among 18 research papers published from 2013 to 2016 that deal 
with big data as a subject, there are no papers written from a public policy point of view. Rather, 
most papers are published based on cognitive computing and cognitive systems theory. In this 
sense, we would like to illustrate the potential and importance of adopting big data informatics 
into public policy decision-making. Following the traditions of policy science, we focus on the 
impact of big data on prediction, which is one of the key factors in making policy decisions. 
Additionally, to analyze how big data informatics can reduce uncertainty and complexity and 
change the process of decision-making as a result, we adopt Herbert Simon’s decision-making 
process model. 

This chapter is organized as follows. First, we briefly review the definition and characteristics 
of big data and, in the second section, introduce several cases using big data in policy decision- 
making worldwide. In the next section, we review Herbert Simon’s decision-making process 
model and illustrate how big data informatics enable us to make decisions within ‘less-bounded 
rationality’ by reducing uncertainty and complexity. Lastly, we introduce several challenges we 
should consider and suggest future directions for implementing big data informatics in public 
policy decision-making. 


Information Tools in the Era of Big Data 


Definition and Characteristics of Big Data 


In their book The Tools of Government in the Digital Age, Hood and Margetts (2007) argued that 
information is often preferred by policymakers because it is cheaper than other policy tools, such 
as authority (sticks or regulations), treasure (carrots or monetary incentives), and organization. 
With the continued advancement of computing power and data availability, governments can 
often easily refer to information as a strong policy tool despite data-related risks and vulner- 
ability, such as biases, manipulation of data, hacking, and privacy violations. Governments are 
expected to use information as a policy tool more smartly and sharply because of the dramatic 
growth of available information and data in different forms and increasing competition for the 
quality of information. 

Thanks to the advent and evolution of big data and machine-learning algorithms, many 
scholars expect how we work and think might be transformed greatly in the near future (Chan 
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& Bennett Moses, 2016), but there is no single clear definition because the type of technologies 
to collect and develop big data, the purpose of using big data, and the platforms big data imple- 
ment vary (Chan & Bennett Moses, 2016) despite this importance. 

As shown in Table 43.1, scholars have defined big data to explain various aspects of big 
data analytics. First, Manyika et al. (2011) focused on the size of the dataset to understand 
the characteristics of big data. Since the size of the big data dataset was too big, it could not 
be handled by a typical database. Snijders et al. (2012) also stressed the size of the big data. 
Dedić and Stainer (2017) argued that big data includes not only structured data but also 
semi-structured and even unstructured data. Even though unstructured data cause a high 
level of handling and trimming cost, with the developed big data dataset and software, we 
can draw meaningful information from various types of data. Targio Hashem et al. (2015) 
focused on the technique and technology that enable us to integrate and analyze diverse, 
complex, and massive datasets. 

A more holistic, comprehensive view has also emerged. To Boyd and Crawford (2012), we 
need to understand various aspects, such as culture, technology, and methodology, to define big 
data analytics because big data analytics is a unity within which all things are combined. IBM 
suggested the concept of 4Vs (volume, variety, velocity, veracity) as four dimensions of big data. 
Volume represents the quantity of generated and stored data. It is estimated that 2.5 quintillion 
bytes of data are created each day (IBM, 2019). Variety means the various sources of data col- 
lection. Thanks to big data technology, we can collect tons of data from various sources — not 
only from classic, traditional sources but also from fast-changing internet and social network 
services. With the adoption of big data technology, various types of data are collected rapidly 
(velocity). According to IBM (2019), 216,000 Instagram posts and 204,000,000 emails sent can 
be collected every 60 seconds. Lastly, veracity represents the certainty of data. Although big data 
analytics solve the problem partially, the problem of certainty of data should be continuously 
addressed in the future because, as IBM (2019) pointed out, an estimated 3.1 trillion USD is 
spent to correct the poor data quality. 


Table 43.1 Various Definitions of Big Data 


Paper Definition and characteristics of Big Data Point 
Manyika et al. Big data is a dataset whose size is beyond the ability of typical Size 
(2011) database software tools to capture, store, manage, and analyze 
Boyd and Big data is a cultural, technological, and scholarly phenomenon Phenomenon 
Crawford based on the ‘interplay’ of (1) technology, which represents a 
(2012) desire to maximize computation power and algorithm accuracy; 


(2) analysis that focuses on identifying patterns to make economic, 
social, technical, and large claims; and (3) methodology 


Snijders et al. Big data includes datasets with sizes beyond the ability of commonly Size 
(2012) used software tools to capture, curate, manage, and process data 
within a tolerable elapsed time 
Dedić and Big data includes unstructured, semi-structured, and structured data Type of data 
Stanier (2017) 
Targio Hashem Big data requires a set of techniques and technologies with new Technique and 
et al. (2015) forms of integration to reveal insights from datasets that are technology 


diverse, complex, and of a massive scale 
IBM (2019) Volume, variety, velocity, and veracity Various aspects 
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Several Cases Using Big Data in Policy Decision-Making 


In 2013, the Washington Post described President Barack Obama as the ‘big data’ president. 
According to the article, since Obama had much faith in the power of big data, he became the 
president who was eager to use it not only for the presidential election but also for health-care 
policy implementation. For instance, in the 2008 presidential election, Team Obama strategi- 
cally used big data to make data-driven decisions (Scola, 2013). First, they organized a team 
of specialists capable of collecting and analyzing big data, such as data engineers and computer 
programmers. This team of engineers collected data from internet databases and social network 
services, such as Google, Twitter, and Facebook. They analyzed them to draw specific patterns 
of voter behaviors. Specifically, they ran more than 62,000 computer simulations and predicted 
their voting behavior (Hilbert, 2011). Based on the results of these big data simulations, Team 
Obama could change the voting behavior of 78% of targeted undecided voters through Face- 
book (Hilbert, 2011). 

After reelection, Obama also stressed the importance of big data— and data-driven evidence- 
based public policymaking. In particular, quantitative evidence drawn from big data analysis was 
used to back up the controversial health-care policy of the Obama administration. In 2013, the 
Obama administration released a massive price list of 3,000 US hospitals in 100 different condi- 
tions to strengthen the health-care policy reform. Revealing the price list of hospitals is not the 
only case of using big data in health care. The US government opened healthcare.gov in 2010 to 
“make data discoverable and make valuable government data available to the public in the hopes 
of better health outcomes for all” (US Healthdata.gov, 2021). 

Guerrero and Lopez (2017) suggested the importance of merged big data in labor economics. 
According to their argument, recently developed big data technology enables us to collect and 
merge macro- and microdata from employers and employees. This fact also leads to new types of 
interactions in the socio-economics field (Taylor, Cowls, Schroeder, & Meyer, 2014). Although 
economists such as Rosen (2004) and Willis (2017) were already aware of the importance of 
merged data through which we can track interactions between individuals and firms, this had 
not yet been realized due to technology limitations. However, since 1998, the consensus has 
been building to solidify economic theory and narrow the gap between theory and practice 
(Guerrero & Lopez, 2017). In this way, employers and employees can exchange labor status and 
information, and the government can implement evidence-based policy. 

Criminology is also an example that shows that big data—driven policies can be implemented. 
For instance, the Korean government successfully pointed out the culprit of the Hwaseong 
serial killer case, which was one of the most notorious serial murder cases, going unsolved for 
more than 30 years. However, in 2019, the Korean government started to compare the DNA 
that culprits left at crime sites with the DNA information of prisoners. As a result, the Korean 
government was able to catch the criminal. 


Policy Decision-Making Process and Bounded Rationality 


Herbert Simon’s Decision-Making Process Model 


As Simon stated in his well-known masterpiece, Administrative Behavior, first published in 1945, 
administrative processes are decisional processes. Simon made key contributions to enhance our 
understanding of the decision-making process. He pioneered the field of decision support sys- 
tems. According to Simon (1959) and his later work with Newell, decision making is a process 
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with distinct stages (Simon, 1959; Newell & Simon, 1972). He posited for the first time the 
decision-making model of human beings. His model of decision-making has three stages: Spe- 
cifically, Simon (1997b) described the process of decisions made: (1) among a given, fixed set of 
alternatives, (2) with (subjectively) known probability distributions of outcomes for each, and 
(3) in such a way as to maximize the expected value of a given utility function (Dequech, 2001). 
Simon developed the decision-making process model in 1997 and suggested the three steps of 
the decision-making process as (1) the listing of all the alternative strategies, (2) the determina- 
tion of all the consequences that follow upon each of these strategies, and (3) the comparative 
evaluation of these sets of consequences (Simon, 1997b, p. 77). He also noted that setting an 
agenda, representing the problem, and discovering and choosing alternatives as components 
(Simon, 1997b, p. 122) should be included in the decision-making theory and argued that the 
stages in the decision-making process require “agenda setting, finding alternatives, selecting 
them, and evaluating them” (Simon, 1997b, p. 127). 

Later generations merged what Simon argued in Administrative Behavior and outlined the 
decision-making process model in a more compact way. As shown in Table 43.2, Asemi, Safari, 
and Zavareh (2011) categorized Simon’s decision-making process model in three steps: the intel- 
ligence stage, design stage, and choice stage. In the intelligence stage, an economic man starts 
to search for problems and collect data to identify what the problem situation is. In the design 
stage, we evaluate possible decision-making alternatives by analyzing both negative and positive 
aspects of each alternative. We can also design possible solution outlines for the problems in this 
stage. The last choice stage is the selection stage. Based on the results drawn from the first and 
the second stages, we select the best solution to solve the problems. 


The Limits of Rationality 


Herbert Simon is most famous for his contribution in adopting the concept of rationality to 
understanding human behavior. As an economist, he tried to describe and predict several kinds 
of economic men (Simon, 1997a) as he believed that economics is a process of investigating 
economic behavior (Heukelom, 2007). To understand why economic actors make specific deci- 
sions, he stressed the importance of rationality. Although Simon himself admitted the difficul- 
ties and complexity of defining rationality (Simon, 1997a), he defined rationality as a process 
insofar as it selects alternatives that are conductive to the achievement of the previously selected 
goals (Simon, 1997b). In other words, to Simon, rationality is concerned with the selection of 
preferred behavior alternatives in terms of some system of values whereby the consequences of 


Table 43.2 Simon’s Decision-Making Process Model 


Stage What We Do Explanation 
Intelligence stage Search for Dealing with problem identification and data collection 
(first stage) problems on the problem 

problem searching + problem formation 
Design stage Develop alternative Evaluating possible alternatives (negative, positive aspects) 
(second stage) solutions Designing solution outlines for the problems 
Choice stage Analyze alternatives Selecting the “best” solution from among the alternative 
(third stage) and choose one solutions using some criterion 


Source: Asemi, Safari, and Zavareh (2011) and Simon (1997b) 
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behavior can be evaluated (Simon, 1990). Since he thought of decisions as a bridge that links 
rationality and human behavior, he developed a theory about the relationship between decision- 
making and the concept of rationality (Barros, 2010). 

Although there are several types of rationality, such as general rationality and procedural 
rationality, Simon focused on bounded rationality as the main subject of his argument (Bar- 
ros, 2010). An economic man is assumed to collect and compare alternatives to make optimal 
decisions. However, as Simon argued, it is impossible for the behavior of a single, isolated 
individual to reach any high degree of rationality (Simon, 1972, 1990) because “the knowl- 
edge that [the individual] is rational is only a small part — almost an insignificant part — of 
the information that we require” (Simon, 1982, p. 214). His intention to be rational leads 
to particular behavior only in the context of conditions in which his behavior takes place 
(Simon, 1972). 


Three Components of Causing Bounded Rationality 


What makes it difficult for an economic man to collect and analyze information so stuck 
into bounded rationality? In his book Administrative Behavior, Simon pointed out incomplete 
knowledge, difficulties of anticipation, and the scope of behavior possibilities as three critical 
components that cause bounded rationality. As shown in the Table 43.3, the first component, 
incomplete knowledge, is closely related to lack of knowledge and information (Simon, 1997b). 
As Simon (1997b) argued, although the concept of rationality requires complete knowledge, in 
reality, it is difficult to have complete knowledge and information due to the limits of time and 
budgets. Rather, what we can have in the real world is fragmented knowledge and information 
(Simon, 1997b). 

Second, difficulties of anticipation can also occur. According to Simon (1997a), the serious- 
ness of actual feelings that people can have through actual experience and the indirect feelings 
from anticipation are drastically different, meaning the same value evaluation system cannot be 
applied in the two cases. Simon argued that actual experience is usually considered more seri- 
ous and desirable than anticipated experience. In this sense, valuation is inevitably limited in its 
accuracy and consistency (Simon, 1997b). 

Third, the component related to the scope of behavior possibilities is related to the physi- 
cal, biological limitations of human beings. Since the number of things that human beings 
can remember and have front of mind is limited due to physical and biological limitations, 
only a very few of all possible alternatives can be considered and conceived as possible 
solutions. 


Table 43.3 Three Components of Causing Bounded Rationality 


Component Explanation 

Incomplete Knowledge of consequence is always fragmentary; lack of information 
knowledge 

Difficulties of Difficulties in anticipating consequences that lie in the future can be only 
anticipation imperfectly anticipated because of lack of experience; feeling valuation is limited 


in accuracy and consistency by the power of individual 
Scope of behavior Only a very few of all possible alternatives ever come to mind due to the biological, 
possibilities physical limitations of human beings 


Source: Simon (1997b, p. 93) 


564 


The Future of Informational Tools 


Potential of Big Data Informatics for Less Bounded Rationality 


Intelligence Stage (First Stage) 


In this section, we are going to analyze how we overcome various components of bounded 
rationality in each stage, using big data informatics as the future policy instrument tool. First, in 
the intelligence stage (first stage), where searching for and identifying the problems is the main 
goal, the first component that causes bounded rationality can be a problem. As we mentioned 
previously, since the knowledge and information we can collect is always fragmented due to 
time and budget limits, it is very difficult to sense the problematic issues and identify what the 
real problem is. 

In this sense, big data informatics can be one considerable solution in overcoming incomplete 
knowledge. First, as we explained, volume, velocity, veracity, and variety are four critical factors 
in understanding the characteristics of big data informatics. These four characteristics show how 
we collect a massive amount of information and knowledge accurately from various sources in 
a very short amount of time. Based on these advantages of big data informatics, we can collect 
data that is necessary to sensing and searching for the problems in the intelligence stage. 

For instance, in January 2020, it was reported that merchants in a nearby wet market fell ill 
from an unknown cause in Wuhan, China. Based on the basic information of symptoms that 
the Chinese government reported to the World Health Organization, the Korean government 
decided to monitor arrivals from China due to the reported symptoms being very similar to 
the SARS variant that hit Korea a few years earlier. Based on cumulated data from the past, the 
Korean government could sense the similarities of SARS and the COVID-19 virus and took 
efficient actions that determined their COVID-19 response success in 2021. This case clearly 
demonstrates the impact of big data informatics in sensing and responding to problems. 


Design Stage (Second Stage) 


In the second stage, what we mainly do is find and develop alternative solutions that enable us 
to solve the problems. As Simon (1997b) noted, all possible alternatives are listed and evalu- 
ated in this stage. Specifically, the determination of all possible consequences that follow each 
of these alternatives should be completed in the design stage. Since all the possible alternatives 
should be listed and the objective, accurate evaluation of values that each alternative has should 
be conducted in this stage, among the three critical components that cause bounded rationality, 
the scope of behavior possibilities and the difficulties of anticipation should be seriously con- 
cerning. Due to the limited scope of behavior possibilities, only a few of all possible alternatives 
can be listed and considered. Also, evaluating both positive and negative aspects of alternatives can 
be insufficient or even manipulated because of difficulties in anticipation that makes it difficult 
for people to set an objective and a clear value evaluation system. 

Adopting big data informatics offers one solution to overcome these problems. First, by 
collecting a massive amount of data faster, big data informatics can solve the incomplete knowl- 
edge problem. Second, by constructing a problem case database that includes various strategies 
adopted to overcome similar problems, we can identify a list of possible alternatives easier and 
faster. Also, by using various big data datasets and techniques, such as a big data—based simula- 
tion, we can evaluate the consequences of possible alternatives in a more objective way. 

For example, in 2020, to flatten the curve of confirmed COVID-19 cases, governments 
around the globe searched various policy responses and tried to evaluate the effectiveness of 
those restriction policies using big data—based simulation techniques. A massive amount of data 
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collected previously were used to anticipate and predict the various types of policy alterna- 
tives. Despite several limitations of current simulation techniques, the big data—based simulation 
technique is widely used and considered an effective tool in the pandemic era (Lee & Moon, 
forthcoming). 


Choice Stage (Third Stage) 


In the last stage, the choice stage, we analyze possible alternatives and choose the best solution. 
In this process, possible future consequences should be analyzed objectively. To do so, choose 
the best solutions that can bring us the most positive future consequences, and then the future 
impacts of those alternatives must be analyzed based on a clear, objective value evaluation system 
and structure using enough data and information, despite the existence of scope of behavior 
possibilities and difficulties in anticipation. Similar to what happens in the design stage because 
of the limitation of scope of behavior possibilities and difficulties in anticipation, it is hard to 
have all the possible alternatives in mind and evaluate the consequences that those alternatives 
can cause objectively. 

But big data informatics provide a way to overcome bounded rationality in the choice stage. 
First, by collecting various cases and constructing a database that includes information about 
possible alternatives and their future impact, it is easy to make a list of problem solutions. Sec- 
ond, big data informatics enable us to select the most plausible, best solution with long-term 
anticipation and prediction based on techniques like big data—based simulations. By anticipating 
the long-term effects and consequences of possible alternatives, we can select the most relevant 
solutions effectively and efficiently. 

Kikstra et al. (2021) simulated the future climate mitigation scenarios that predict the impact 
of persistent COVID-19-telated energy demand changes by 2025. Specifically, Kikstra et al. 
(2021) categorized future scenarios into four groups and predicted what will happen in the 
future using cumulated data around the world. Like this example, by adopting big data informat- 
ics, we can overcome the limited scope of behavior possibilities and difficulties in anticipation 
and easily evaluate the future impact from long-term perspectives. We summarize the character- 
istics of related components of bounded rationality and strategies using big data informatics to 
overcome these bounded rationalities within Simon’s (1997b) decision-making process model 
stage (see Table 43.4). 


Remaining Challenges 


Privacy and Ethics 


Although it is hard to deny the potential of big data as a new policy tool that expands our 
‘bounded rationality, some challenges still remain. The first challenge encompasses ethical, pri- 
vacy-related issues. Especially among various types of big data, individual-related big data should 
be handled very carefully (Dunleavy, 2016). For instance, as Heitmueller et al. (2014) argued, 
health information is different from other types of big data because it includes sensitive informa- 
tion related to privacy. In particular, if a person’s health information is released unintentionally, 
it might cause embarrassment and financial harm (Heitmueller et al., 2014). Despite the impor- 
tance of protecting privacy and sensitive information, technological, normative, and legal ways 
to save sensitive data safely have not yet been prepared. As Boyd and Crawford (2012) noted, the 
fact that many data are publicly available does not mean that all the analyses are legitimate, and 
many users use big data without specific norms or responsibilities. 
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Table 43.4 Adoption of Big Data and Overcoming Bounded Rationality 


Stage What We Do Related Bounded Overcoming Strategies Using Big 
Rationality Components Data 
Intelligence stage Searching for Incomplete knowledge Sensing/collecting issue cases 
(first stage) problems Collecting data 
Design stage Developing Incomplete knowledge Collecting data 
(second stage) alternative Scope of behavior Constructing a database 
solutions possibilities Collecting cases 


Difficulties in anticipation Evaluating possible alternatives 
using anticipation and 
prediction data and 
techniques (e.g., simulation) 

Choice stage Analyzing alternatives Incomplete knowledge Collecting data 

(third stage) and choosing one s Sou a Steen 

one Difficulties in anticipation Anticipating long-term effects 

Considering future 

and consequences 

consequences f f 

Future simulation 

Selecting the most plausible, best 
solution based on anticipation 


and prediction results 


Source: The author 


Additionally, defining the scope of privacy varies by culture and country. As Dunleavy (2016) 
pointed out, under Anglo-American democracies, taxation is a personal issue that should be 
protected while tax submissions are public documents in Norway (see Devos & Zackrisson, 
2015). In this sense, we should consider how to protect privacy and ethics and think about to 
what extent we should protect privacy. Additionally, in the era of big data, people are increas- 
ingly tracked and monitored by the government and private firms (Michael & Miller, 2013). 
Resultingly, can we see tracking employees’ every move and measuring their performance inces- 
santly as an ethical phenomenon? As Michael and Miller (2013) argue, this phenomenon can be 
an ethical challenge that hurts the human spirit. 


Quality of Data 


Another challenge we should consider is the quality of the data. As pointed out earlier, the types 
and sources of big data vary. Big data can be produced intentionally and unintentionally. It can 
be produced by various actors from the government to giant IT conglomerates. Additionally, 
although text is the most popular type of big data, voice records or video can also be analyzed 
as such. We even analyze various types of merged data using developed big data technology. In 
sum, collecting raw data is not as difficult these days as it was in the past. The important thing is 
not how to collect and produce big data but rather how to produce well-structured data or trim 
and reform unstructured data to structured data efficiently. 

Constructing structured data is closely related to the quality of experts, as highly-skilled 
experts can trim unstructured data efficiently to use them effectively. In this sense, according to 
Dunleavy (2016), although highly-skilled expertise is necessary for implementing big data-based 
policy-making, it is hard to find such staff to work for governments or universities. Additionally, 
since most of the major projects related to big data are done by giant IT firms such as Google, 
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Amazon, or Apple, the government and public organizations face difficulties finding relevant 
staff (Dunleavy, 2016). In order to attract highly-skilled experts, the public sector, including 
governments, should consider attractive intrinsic and extrinsic incentives. 


Problems of Democracy and Constructing a Multilateral Structure 


Since understanding and analyzing big data require a complex methodology and a high level 
of technological knowledge, adopting big data in policy decision-making can be the exclusive 
property of highly-skilled technocrats. As Lavertu (2016) argued, this can cause some very seri- 
ous concerns. 

One possible solution to pursue both effectiveness and democracy simultaneously is con- 
structing a multilateral participating system. Lane (2016) suggested a ‘quadruple helix’ to use 
big data in public policy to increase the effectiveness and efficiency of big data policies. There 
are four actors in the quadruple data helix: state and city agencies, universities, private data pro- 
viders, and federal agencies. Each actor has its own responsibility. First, state and city agencies 
address immediate policy issues and produce policy data. Since local governments encounter 
citizens at the street level, they can be proper data producers. Universities should be the third 
party that processes, stores, analyzes, and disseminates data. Universities must train specialists 
who not only collect and analyze the data but also predict and oversee the future from a macro 
point of view. The third actor, private data providers, should produce more structured, well- 
organized data that can reduce the time needed for trimming. Last, the federal or central gov- 
ernment can provide a macro strategy based on a long-term perspective. In particular, the central 
government must try to produce and collect data continuously from a long-term perspective. As 
Maclennan and Moore (1999) reported, researchers have incessantly criticized the lack of data 
in predicting policy decision alternatives. In this sense, the central government must be the one 
who invests resources not only to researchers but also to practitioners who face the challenges 
related to policy decision-making. 
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THE FUTURE OF BEHAVIOURAL 
TOOLS IN PUBLIC POLICY 


Stuti Rawat and Michael Howlett 


Although there is increasing awareness and use of behaviourally informed approaches and tools 
in public policy, there is only limited systematic evidence on developments across the world 
and across sectors. Possessing knowledge about the spread and nature of behavioural policy, 
however, is a needed first step towards better engaging with the challenges confronting behav- 
iourally inspired approaches and developing strategies to overcome them. This chapter identifies 
the key challenges associated with the adoption of behavioural policy tools and assesses how far 
behavioural policy has attempted to engage with these issues. Towards that end, we undertake 
a bibliometric analysis of the top four public policy journals between 1990 to 2016 to identify 
the research trajectory of behavioural studies that have been featured in them, unpacking their 
policy areas, content and geographic focus. We conclude by discussing future research directions 
for behavioural policy tools. 


Introduction 


Even as the increased awareness and use of behaviourally informed approaches and tools in 
public policy (OECD 2017a; Sanders, Snijders & Hallsworth 2018) is being challenged with 
criticism directed towards different facets of it (see, e.g., (Lepenies, Mackay & Quigley 2018; 
Kuehnhanss 2019)), there exists limited systematic evidence on how behavioural policy tools 
have been used around the world and across sectors. The literature that synthesizes such devel- 
opments has typically been in the form of review studies, surveys or reports that have restricted 
their focus to certain sectors, such as health (Matjasko, Cawley, Baker-Goering & Yokum 2016; 
Vlaev, King, Dolan & Darzi 2016), energy and environment (Baddeley 2012; OECD 2017b), 
climate (Pollitt & Shaorshadze 2012; Howlett & Rawat 2019), and poverty and development 
(Anand & Lea 2011; Bouguen, Huang, Kremer & Miguel 2019). 

Knowledge about the spread and nature of behaviourally informed policy is the first step 
towards assessing where it has lagged — sectorally and geographically — and what challenges it 
still needs to address. Thus, helping shed light on the more practical questions of where, when, 
how and to what extent behavioural sciences can be used in the policy process (Ewert 2020) 
and their future. 

In order to map the prevalence of behavioural public policy, we direct our attention to the 
public policy literature with the underlying assumption that it reflects developments on the 
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ground, as well as playing a role in shaping them (Bavel, Herrmann, Esposito & Proestakis 2013; 
Newman, Cherney & Head 2016). 

Within the public policy literature, we focus on the top four journals in the discipline to 
identify the research trajectory of behavioural studies that have featured in them over the last 
three decades. We restrict our analysis to the top-ranked journals for two reasons. First, focusing 
on a finite number of journals allows us to benchmark behavioural studies against the total num- 
ber of studies that have been published in these journals during the same period, thereby giving 
a sense of their relative numbers. This is in contrast to bibliometric surveys that rely on large 
databases such as the Web of Science or Google Scholar, which are useful in reflecting only the 
absolute number of publications on a certain topic (e.g., (Weible, Sabatier & McQueen 2009). 
Second, notwithstanding the criticism directed at journal rankings (Bornmann 2013; Lariviére & 
Sugimoto 2019), the top journals in any discipline tend to be particularly influential within the 
scientific community in terms of their perceived quality (Adam 2002) and in attracting a high 
number of submissions (Gaston, Ounsworth, Senders, Ritchie & Jones 2020). Thus, playing an 
integral role in driving forward the evolution of the discipline as well as reflecting what might 
be crudely labelled as the mainstream view. 

Based on this sample, our analysis aims to answer two research questions. First, what are 
the temporal, spatial and sectoral spread of behavioural studies? Second, are behavioural studies 
engaging with the challenges associated with the behavioural approach? The answers to these 
questions provide us with insights about the evolution of behavioural public policy, the research 
gaps that remain and its future. 

In what follows, we use the terms behavioural policy, behavioural approach and behavioural policy 
tools interchangeably. This chapter unfolds in four parts: first, we discuss the seven most fre- 
quently cited challenges with respect to behavioural policy. Next, we discuss the empirics of this 
article, focusing on the method and data. This is followed by a discussion of our results. We end 
with a brief conclusion. 


Seven Key Challenges to Behavioural Policy Research 


Based on the larger literature, we identify seven key challenges most frequently associated with 
behavioural policy. The first challenge is the lack of evidence on long-term effects. Sanders et 
al. (2018) attribute this to the general absence of longer-term studies and projects. Goodwin 
(2012), however, traces this to the nature of nudges themselves. He argues that nudges overem- 
phasize individual preference and adopt an atomistic approach to social structure, resulting in 
their limited ability to usher in long-lasting behavioural modifications (Mills 2013). 

The second challenge is that the behavioural approach ignores cultural and national differ- 
ences, and behavioural insights tend to be tailored to the Western context (Levinson & Peng 
2006). It comes as no surprise that behavioural public policy as manifested through nudge units 
and related initiatives is found to be driven by Anglo-American countries (John 2019). Henrich, 
Heine and Norenzayan (2010) use the term WEIRD — Western, educated, industrialized, rich 
and developed — to describe the vast majority of those who are engaged in generating behav- 
ioural insights and their subjects. They argue that behavioural scientists need to engage with a 
wider pool of (non-Western) subjects if they are to generate behavioural insights that are appli- 
cable to other cultures and contexts. 

A third challenge associated with the behavioural approach in public policy is the fact that 
policymakers are themselves behavioural agents and hence, not immune to psychological biases 
(Viscusi & Gayer 2015). As Sunstein puts it, ‘[F]or every bias identified for individuals, there is 
an accompanying bias in the public sphere’ (Sunstein 2014a:102). For instance, ‘confirmation 
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bias’, which is the selective gathering of evidence to support previously held beliefs (Nicker- 
son 1998), is frequently identified with the behaviour of policymakers. This ‘rationality para- 
dox’ characterizes policymaking when policymakers who themselves have bounded rationality 
employ nudges (Lodge & Wegrich 2016). 

A fourth challenge identified with behavioural policy is the possibility of behavioural spill- 
overs and unintended consequences (Dolan & Galizzi 2015). For instance, energy efficiency 
interventions are associated with a rebound effect, whereby consumers end up increasing prod- 
uct use when their energy costs fall (Miller & Mannix 2016). Similarly, Costa and Kahn show 
that an energy conservation nudge (in the form of feedback on one’s own and their peers’ 
electricity usage) resulted in a boomerang effect of an increase in electricity consumption by 
certain target groups (Costa & Kahn 2013). Moreover, the interaction of behavioural tools with 
the available and often-competing policy instruments (see Howlett 2005, 2000; Oberg, Lun- 
din & Thelander 2015) can influence the policy outcome in unintended ways (see Fonseca & 
Grimshaw 2017; Fanghella, Ploner & Tavoni 2021). 

The fifth challenge relates to the lack of agreement about which behavioural insights should 
be targeted when and which insight has the strongest impact (Lunn 2013), and, at a more funda- 
mental level, what constitutes a behavioural insight (Foster 2017). Behavioural insights comprise 
a wide range of different practical approaches and different normative starting points (Schubert 
2017; Kuehnhanss 2019), resulting in a wide range of definitions for behavioural insights that are 
often potentially contradictory and too restrictive (see Lunn 2012, 2014; Hansen 2016). 

The sixth challenge has to do with the fact that the behavioural approach has little to say 
with regard to organizations. Nudges have tended to use the individual as their unit of analy- 
sis, and so not much is known about how behavioural science can improve and complement 
regulation in organizations (Sanders et al. 2018), and there is limited evidence of how it can 
shape the environment in which organizations operate (see for instance, Frey & Eichenberger 
1994; Barr, Mullainathan & Shafir 2012). An OECD background paper notes that the reliance 
of behavioural policy on the RCT methodology hinders its use in the context of organizational 
phenomena that do not always lend themselves to RCTs (Foster 2017). Thus, there are fewer 
cases evaluating the impact of behavioural interventions on organizations than on individuals 
(Shephard 2017). 

The seventh challenge to the behavioural approach as embodied in nudges is on ethical 
grounds. Goodwin (2012) considers nudges to be manipulative as they basically exploit imper- 
fections in human judgment. Other scholars articulate concerns regarding the risk of infantiliz- 
ing and decreasing the autonomous decision-making capacity of people because of nudging 
(Bovens 2009; Hausman & Welch 2010). Rizzo and Whitman (2009) question the basis on 
which the values and preferences sought to be promoted through nudges are determined. Typi- 
cally, choice architects project the values and preferences of their conceptions onto policy tar- 
gets, but these projections may not even be in line with what targets actually prefer to choose 
(Qizilbash 2009; Selinger & Whyte 2011). Nudging, it is argued, could encourage abuse of 
power by technocrats (Rebonato 2012). 


Data, Method and Methodological Issues 


The data for this analysis came from behavioural studies published in the top four public policy 
journals between 1990 to 2016. These journals were identified using three commonly used 
ranking systems — Google Scholar Metrics (Google Scholar Metrics 2016), SCImago Jour- 
nal rankings (SJR) (SJR 2016) and Web of Science rankings by Thomson Reuters (Thomson 
Reuters 2016) and their ranking of journals under the subject ‘Public Policy’ and/or ‘Public 
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Administration’. Google Scholar listed ‘Public Policy and Public Administration’ as a discipline. 
The SCImago Journal rankings and the Web of Science rankings did not have a separate cat- 
egory for ‘Public Policy’ but instead, under the discipline of ‘Public Administration’, listed both 
public policy and public administration journals. 

For every journal that appeared in the respective annual rankings of SJR and Web of Science, 
a mean rank was calculated (by dividing the sum of the journal’s ranks by the number of years 
it featured in the respective ranking systems. This was not done for Google Scholar rankings as 
they were not available annually. At the time of data collection, Google Scholar provided a single 
set of ranks which was based on the activities of the last five years). By arranging these mean rank 
values in ascending order, a new adjusted ranking for journals was determined for both SJR and 
Web of Science. This allowed us to identify the top ten journals as per SJR and Web of Science, 
respectively. We then shortlisted the journals that featured in the top ten of our adjusted ranks 
for SJR and Web of Science and the top ten Google Scholar rankings. 

Four public policy journals featured among the top ten for each of the three sources. These 
were Journal of Public Administration Research and Theory, Governance, Public Administration, and 
Public Administration Review. Between 1990 to 2016, these four journals together produced 4,578 
articles (including research articles, commentaries and editorials but excluding book reviews 
and commemorative and memoriam pieces). A key word search of the title and abstract using 
the words ‘behaviour*, experiment*, prospect, random*, nudge*’ and variants of their spell- 
ings resulted in a database of 116 studies. Using a decision rule, which we will discuss shortly, 
we reduced the database to a shortlist of 101 behavioural studies. Establishing a decision rule 
brought to the fore definitional issues that characterize behavioural policy at large. 

Finally, to answer the research questions, each of the 101 behavioural studies was individually 
examined. For establishing the geographic spread of these studies, the site of the behavioural 
study (and not the location of the author) was examined. In case of multiple-country studies, 
each country was regarded as a separate data point. For analyzing the sectoral spread of studies, 
following a perusal of the articles, 20 different policy areas and an additional ‘miscellaneous’ cat- 
egory for those topics which did not belong to any of these were identified. Each behavioural 
study was then tagged under one or more of these categories. 


Definitional Issues: Ambiguity Regarding What Constitutes 
‘Behavioural’ Policy 


According to Galizzi (2014), among practitioners and researchers, there is some debate on what 
really is ‘behavioural’ policy. It is not clear whether behavioural refers to methods or insights or 
both. There is a tendency among practitioners to define a policy as behavioural because of its 
use of experiments (in the laboratory or field) or randomized controlled trials (see, for instance, 
(Haynes, Goldacre & Torgerson 2012; Bavel et al. 2013). Many researchers, however, consider 
such framing to be problematic (Lepenies et al. 2018). 

Our decision rule adopted a wide umbrella in judging whether a study classified as behav- 
ioural. It was classified as such if it employed a behavioural insight and/or an experimental 
approach. We chose to include experimental methods under the behavioural category because 
in the policy world, these are widely included under the behavioural umbrella (see, for instance, 
(Haynes et al. 2012). 

Roughly a fourth of the articles qualified as behavioural by focusing on a behavioural insight 
making use of the experimental methodology. However, half the studies qualified as behavioural 
based on the methodological criterion alone. The remaining fourth were classified as behav- 
ioural on the grounds of dealing with a behavioural insight. 
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The other definitional issue we encountered was determining how close to the concep- 
tual core of ‘behavioural insights’ policy tools needed to be for them to qualify as behavioural 
(Selinger & Whyte 2011; Galizzi 2014). For instance, Hood (1986)’s ‘NATO’ taxonomy of 
policy tools includes nodality (or information), along with authority, treasure and the organi- 
zational resources of government. Thus, the information-based policy instrument predates the 
‘information’ nudge (Sunstein 2014b). But does this mean that every time the government used 
an information-based policy tool, it was actually a behavioural nudge? Hausman and Welch 
(2010) think not. They use Thaler and Sunstein’s example of the Ambient Org as a behavioural 
nudge that provides feedback on energy consumption which helped save energy (Thaler & 
Sunstein 2008) to argue that reminding people to think about energy use is no different from 
warning labels on cigarettes, educational campaigns or signs reminding people to drink more 
water on hot days. Selinger and Whyte (2011) argue in a similar vein that some interventions 
that are tagged as a ‘behavioural’ may not necessarily be so (‘mistaken nudges’), and there may 
also exist borderline cases where it is not clear whether the intervention qualifies as a nudge or 
not (‘fuzzy nudges’). 

To reference the ‘provision of information’ example to our analysis, we found ten behavioural 
studies that dealt with this. In cases where the authors did not explicitly employ terminology 
from the behavioural sciences, it became difficult to categorize which study dealt with the 
provision and use of information in the manner of a behavioural insight and which dealt with 
information as a tool like any other. Here again, we adopted a broad lens in considering the 
study as behavioural. 


Results and Discussion 


Behavioural Policy Studies Have Moved From a Cyclical Trajectory 
to an Ascendant Path 


The database of 101 behavioural studies represented roughly 2% of all the articles that were 
published in the four public policy journals between 1990 to 2016. However, this figure masks 
the changes in the temporal trajectory taken by behavioural studies in policy literature. Between 
1990 and 1999, about 1% of articles in the top policy journals were behavioural. And between 
2000 and 2009, behavioural studies featured even less in this policy literature, constituting only 
0.8% of all published articles. However, between 2010 and 2016, behavioural articles constituted 
almost 5% of articles that were published in the top public policy journals. 

Breaking down these decadal averages into annual values, Figure 44.1 depicts the number and 
share of behavioural studies as a percentage of the total number of studies that were published 
in the top public policy journals for every year from 1990 onwards. The clustered bars represent 
the absolute number of behavioural studies every year while the dashed line depicts the relative 
share of these behavioural studies. 

Between 1990 to 2010, we see a cyclical pattern to how behavioural articles feature in these 
journals, with periods of increase (such as 1990—1992) followed by periods of decrease (such as 
1992-1996). However, starting from 2007 onwards, the cyclical pattern gives way to a rising 
trend. Post 2010, we see a discernible increase in the absolute number as well as the relative 
share of behavioural articles in the top policy journals. By the year 2016, behavioural studies 
comprised a tenth of all public policy articles. This figure is similar to that found in the three top 
economics journals in 2016, where behavioural studies comprised 8% of all published articles 
(Rawat 2019). 
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Figure 44.1 The Annual Number and Share of Behavioural Studies from 1990 to 2016 in the Top Public 
Policy Journals 


Source: The authors’ calculations 


The spread of behavioural studies in the policy literature starting from the second half of 
the 2000s reflects the increasing awareness and acceptance of behaviourally informed insights 
and tools in public policymaking and the appearance of Thaler and Sunstein’s pivotal work on 
nudges (Thaler & Sunstein 2008). The boost given by this publication, according to Oliver 
(2013), fell on fertile ground as the 2008 financial crisis played an instrumental role in bringing 
to the fore the failings of the neoclassical models (Thaler & Mullainathan 2008). 

Around the same time (i.e., post 2008), Cameron, Mishra and Brown (2016) also found that 
in the development sector, the use of randomized controlled trials (RCTs) also took off. The 
fact that many of these RCTs were premised on behavioural interventions (see, for instance, 
(Duflo, Kremer & Robinson 2011; Kremer, Lee, Robinson & Rostapshova 2013; Drexler, 
Fischer & Schoar 2014) helped provide an additional boost to the use of behavioural insights in 
crafting policy. 

A relatively restricted geographical spread: About half of all behavioural policy studies are 
focused on the US, although in recent years, other regions are making forays. The geographic 
spread of behavioural policy studies is not widespread. Figure 44.2 presents a temporal mapping 
of behavioural studies between 1990 and 2016 by location. Until 2009, almost all behavioural 
studies were set in the US. From 2010 onwards, however, behavioural studies began to cast a 
relatively wider net in terms of geographic location, although as a single country, the US still led 
with a third of all behavioural studies published between 2010 and 2016 set there. 

In terms of regions, the highest proportion of behavioural studies during this time were 
located in Europe (47% of all behavioural studies), across Belgium, Denmark, Germany, Italy, 
Netherlands, Poland and the UK. Among the European nations, Denmark (18%), the UK (12%) 
and the Netherlands (9%) appear to have been the leaders in behavioural policy studies. Other 
European countries featured one or two behavioural studies during this period (representing 1% 
to 2% of all behavioural studies). 

The remaining behavioural studies were scattered across continents. Altogether, three studies 
were set in South America in Colombia and Bolivia. Three studies were set in Asia, with one 
each located in Israel, South Korea and Taiwan (China). One study each was located in Uganda 
and Mexico. 
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Figure 44.2 A Temporal Mapping of Behavioural Studies by Region 


Source: The authors’ calculations 


The overwhelming representation of the US and Europe in behavioural studies may be rooted 
in the general tendency of the discipline of public policy to be focused on Western countries 
(Rawat 2019), a phenomenon which is not unsurprising given its historical development and 
may be tied to the home location of the top four journal examined. But the gradual geographic 
spread of behavioural studies post 2010 can also be linked to method. That is, while behavioural 
studies in public policy in the 1990s and 2000s were synonymous with laboratory experiments, 
post 2010, there has been an increase in the use of field experiments, quasi-experiments, natural 
experiments and internet-based randomized surveys (using online research panels). 

This diversification in methods vis-a-vis behavioural studies in public policy may perhaps 
have been rooted in an osmosis from other fields such as political science (Druckman, Green, 
Kuklinski & Lupia 2006) and economics (Levitt & List 2009; Webber & Prouse 2018; Bouguen 
et al. 2019), with the growing influence of experiments in different forms allowing for more 
geographical dispersion than previously. 


Sectoral Spread: Performance Is a Key Topic of Behavioural Policy Studies 


In Figure 44.3 a word cloud is presented based on the most frequent terms found in the titles 
and abstracts of the behavioural studies in the four top public policy journals examined. The size 
of each word denotes the frequency with which it appeared. Certain words leap out from the 
word cloud (for example, ‘performing , ‘experiments’, ‘behavior. 

In terms of policy areas, performance in its myriad forms (e.g., performance management, 
performance motivation and so on) was a key topic of study. While behavioural studies have been 
used in sectors like health, education, finance, law and order, environment and natural resources, 
there are no studies that deal with agriculture, climate change, water, transport or energy — areas 
in which behavioural insights have been applied in the policy world (OECD 2017a, 2017b). 
However, behavioural studies in policy literature have been used to explore more abstract issues, 
such as those related to the decision-making processes of policymakers: issues related to trust, 
risk, satisfaction, attitudes, beliefs, identity and diversity. Typical public policy areas that behav- 
ioural articles have studied include public service motivation, decentralization, public service 
delivery and participation (including collaboration, cooperation and co-production). 
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Figure 44.3 Word Cloud Based on the Most Frequent Terms in the Titles and Abstracts of Behavioural 
Studies in the Top Public Policy Journals 


Behavioural Policy Research Is Engaging With Some 
But Not All Challenges 


The analysis presented earlier shows that behavioural studies in public policy are making forays 
into non-Western contexts where they are increasingly being practiced. This is evident from the 
studies that were set in South Korea, Uganda and Taiwan. Thus, behavioural public policy lit- 
erature is making an attempt to address the challenge of cultural variation (the second challenge). 

The sectoral mapping of behavioural studies revealed that there are numerous studies related 
to the decision-making processes of administrators, managers and politicians. Thus, behavioural 
studies in public policy are very cognizant of the psychological biases that policymakers exhibit 
and are deeply invested in exploring this issue. 

Although there is still no clear sense of which nudges are most effective and which can be 
used when, there were examples of behavioural studies exploring different types of nudges. This 
engagement, even if piecemeal, can contribute towards building a better understanding of the 
different behavioural insights. 

According to Foster (2017), behavioural approaches can be applied to organizations either 
by directly intervening with organizational procedures, processes and policies or by influenc- 
ing individuals in the organizations. While no behavioural studies in the top policy journals 
dealt with organization-level behavioural interventions directly, there were limited examples of 
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Table 44.1 The Challenges of Behavioural Public Policy and Whether Behavioural Studies in Policy 
Literature Have Engaged With Them 


Challenges of Behavioural Public Policy 


Long-Term Cultural Biases of Spillovers When to Use Nudging Ethics of 


Effects Variation Policymakers Which Nudge Organizations Nudges 
[1] [2] BI [4] [5] [6] [7] 
Has behavioural public x y y x y y x 


policy explored this? 


(The ‘x’ denotes a no, and the ‘y’ denotes a yes.) 


studies that focused on individuals in the organization, including leaders, thus shedding light on 
aspects of the application of the behavioural insights at the organizational level. 

In contrast, no evidence was found of behavioural studies exploring the long-term impact of 
behavioural interventions (the first challenge), spillovers (the fourth challenge) or the ethics of 
nudging (the seventh challenge). These overall results are depicted in Table 44.1. 


Conclusion 


Despite the rising prominence of behavioural policy instruments, there has been limited system- 
atic evidence of the research agendas and activities around them. Through a bibliometric analysis 
of the top ranked public policy (and/or public administration) journals, we traced the evolution 
of behavioural policy research over time and across regions. 

We identified seven key challenges that have been associated with behavioural policy and 
found that the behavioural studies in our sample engaged with four of these challenges (cultural 
variation, biases of policymakers, which nudges work and when and organizational nudges) to 
varying extents. However, there were no studies exploring the remaining three challenges — 
spillovers, long-term impact of behavioural interventions and the ethics of nudging. Given our 
limited sample, these results should be interpreted with nuance in that an absence of studies 
actually indicates a lower level of engagement rather than a complete lack. Regardless, more 
research is needed on these three aspects as well as the remaining four challenges, if the barriers 
posed by them are to be overcome. 

In recent times, some emerging concerns with the behavioural approach have come to the 
fore, such as the relatively modest effects of behavioural policy tools when put into practice on 
the ground (DellaVigna & Linos 2020) and their failure to scale up (Léschel, Rodemeier & 
Werthschulte 2020). Researchers argue that the average impact of a nudge in academic literature 
has tended to be higher because of publication bias and exacerbated by low statistical power 
(DellaVigna & Linos 2020), with effects often bound to the short run (Antinyan & Asatryan 
2019). If behavioural policy is to continue on its ascendent path, then it will need to address 
this criticism while accepting that behavioural policy tools may not suffice as stand-alone policy 
instruments (Damgaard 2020). 

We end this chapter by highlighting that scholars have noted the development of ‘behavioural 
public policy’ (which focuses on the empirical effects of behavioural insights, their categoriza- 
tion and other related issues) and “behavioural public administration’ (which focuses on cogni- 
tive and decision biases amongst bureaucrats and relations between bureaucrats and citizens) as 
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separate sub-fields that rarely speak to each other (Ewert, Loer & Thomann 2021). However, 
greater integration between them may be desirable as it can deepen understanding of different 
phases of the policy process (Gofen, Moseley, Thomann & Kent Weaver 2021) and can aid the 
use of behavioural policy tools alongside traditional policy instruments. 
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